search

It's a pretty popular search command and it is used in all sorts of situations. Below are some really cool searches that use search along with other search commands.

Search Golf - Episode 1

# source the events in chron order (so "start" is before "end")
index=cst sourcetype=mav-golf | reverse 
# add a line number / temp id to the events
| eval lc=1 | accum lc 
# extract a field to make it easier to deal with action
#  not really necessary in this example - could just search for "start" / "end"
| rex field=_raw "ID=\S\s(?<action>\S+)\s" | stats list(action) as action by ID, lc 
# find action=start for each identifier and join that back into each row
| join ID type=left [search index=cst sourcetype=mav-golf | reverse | eval lc=1 | accum lc | rex field=_raw "ID=\S\s(?<action>\S+)\s"  | search action=start | stats first(lc) as open by ID] 
# find action=end for each identifier and join that back into each row
| join ID type=left [search index=cst sourcetype=mav-golf | reverse | eval lc=1 | accum lc | rex field=_raw "ID=\S\s(?<action>\S+)\s"  | search action=end | stats last(lc) as close by ID] 
# lastly, test each event to see if it's own id is between the start and end.
#  if so - count it.
| eval sc = if(lc>open, if(lc<close, 1, 0), 0) 
# And then sum up those events which should be counted.
| stats sum(sc) as num_events by ID

purpose:

Find the number of events within a sequence of events based on a shared identifier. Keywords ("start" and "end") mark the beginning and end of the sequence. The search cannot use the transaction command.

requirements:

Data like the following: 01/01/2014 01:01:00.003 ID=a start blah blah 01/01/2014 01:01:01.003 ID=d more blah blah 01/01/2014 01:01:02.003 ID=a end blah blah 01/01/2014 01:01:03.003 ID=b start blah blah 01/01/2014 01:01:04.003 ID=c start blah blah 01/01/2014 01:01:05.003 ID=y more blah blah 01/01/2014 01:01:05.006 ID=c more blah blah 01/01/2014 01:01:05.033 ID=c more blah blah 01/01/2014 01:01:06.003 ID=c end blah blah 01/01/2014 01:01:06.033 ID=b more blah blah 01/01/2014 01:01:07.003 ID=b end blah blah 01/01/2014 01:01:08.004 ID=c more blah blah 01/01/2014 01:01:09.005 ID=b more blah blah

comments:

Authentication Anomalies via "area" algorithm

`authentication` 
| search ( action="success" ) 
| eval citycountry=src_city+", "+src_country 
| stats (name, purpose, rqts, searchstring, created_at, updated_at, commentary) VALUES(citycountry) as CityCountry(name, purpose, rqts, searchstring, created_at, updated_at, commentary) VALUES, dc(citycountry) as loccount, max(src_lat) as maxlat, min(src_lat) as minlat,max(src_long) as maxlong, min(src_long) as minlong by user 
| eval delta_lat = abs(maxlat-minlat) 
| eval delta_long=abs(maxlong-minlong) 
| eval area= delta_lat * delta_long * loccount 
| where area > 1000

purpose:

Use 'area' to identify whether a given person could travel the distance between login events.

requirements:

ES app (or something with a matching macro

comments:

Indexes my user can search.

| rest /services/data/indexes
| search
    [
    | rest /services/data/indexes
    | dedup title
    | table title
    | search
        [
            | rest splunk_server=local /services/authorization/roles
            | search
                [
                        | rest splunk_server=local /services/authentication/users
                        | search
                            [
                                        | rest /services/authentication/current-context
                                        | search type=splunk
                                        | table username
                                        | rename username as title ]
| fields roles
| mvexpand roles
| rename roles as title] imported_srchIndexesAllowed=*
| table imported_srchIndexesAllowed
| rename imported_srchIndexesAllowed as title
| mvexpand title] ]
| stats values(splunk_server) as splunkserver by title
| eval splunkserver=mvjoin(splunkserver,":")
| lookup non_internal_indexes title as title OUTPUT description as description
| fields title, description, splunkserver
| rename title AS Index

purpose:

requirements:

lookup csv with columns "title" of Index and "description"

comments:

We use in a ~200 Users env to show the user in which Index he is allowed to search.

User Search Restrictions and Last Logins

| rest /services/authentication/users splunk_server=local
| table title realname email roles 
| mvexpand roles 
| join roles type=outer 
    [| rest /services/authorization/roles splunk_server=local
    | fields imported* title srch* 
    | fields - *Quota *TimeWin *capabilities 
    | rex mode=sed field=srchFilter "s/^\*$/true/" 
    | rex mode=sed field=imported_srchFilter "s/^\*$/true/" 
    | eval search_restrictions=if(imported_srchFilter="","( ".srchFilter." )",if(srchFilter="","( ".imported_srchFilter." )","( ".srchFilter." ) OR ( ".imported_srchFilter." ) ")) 
    | fields - srchIndexesDefault 
    | eval srchIndexesAllowed_new=mvjoin(srchIndexesAllowed," OR index=") 
    | eval index_restrictions="( index=".srchIndexesAllowed_new." )" 
    | table title search_restrictions index_restrictions 
    | rename title AS roles ] 
| stats values(roles) AS roles values(search_restrictions) AS search_restrictions values(index_restrictions) AS index_restrictions by title,realname,email 
| rex mode=sed field=search_restrictions "s/\( \)//g" 
| eval search_restrictions=mvjoin(search_restrictions," OR ") 
| eval index_restrictions=mvjoin(index_restrictions," OR ") 
| eval realname=upper(realname)
| rename title AS username
| eval restrictions=if(search_restrictions="( )",index_restrictions,search_restrictions." ".index_restrictions)
| table username email username roles restrictions
| search username=* 
| rex mode=sed field=restrictions "s/^ OR //g" 
| join type=outer username 
    [ search (index=_audit "login attempt info=succeeded) OR (index=_internal "GET /splunk/en-US/account/login") 
    | stats max(_time) AS "last_login" by user 
    | rename user AS username ] 
| eval days_ago=floor((now()-last_login)/86400) 
| eval last_login=strftime(last_login,"%Y-%m-%d %H:%M:%S") 
| fillnull last_login value="Never"

purpose:

requirements:

comments:

Shows a table showing the inherited search and index restrictions for each user, represented as an SPL string. In certain cases it may not be 100% accurate, but it covers most instances that I have run into. NOTE: does not account for `searchFilterSelecting = false` defined in authorize.conf

Show links between tags, sourcetypes, apps and datamodels.

| rest splunk_server=local count=0 /servicesNS/-/-/admin/datamodel-files 
| spath input=eai:data output=base_search path=objects{}.baseSearch 
| spath input=eai:data output=constraints path=objects{}.constraints{}.search 
| eval tag_content = mvappend(base_search,constraints) 
| rex max_match=0 field=tag_content "tag=\"?(?<tag_name>\w+)\"?" 
| mvexpand tag_name 
| rename title AS datamodel 
| append 
    [| rest splunk_server=local count=0 /servicesNS/-/-/admin/eventtypes 
    | rename eai:acl.app AS app tags AS tag_name 
    | search app="*TA*" 
    | rex max_match=0 field=search "sourcetype=\"?(?<sourcetype>[^\s\"^)]+)\"?" 
    | mvexpand sourcetype 
    | mvexpand tag_name 
    | eval app_sourcetype=mvzip(app,sourcetype,"__") 
    | stats list(tag_name) as tag_name by app, sourcetype,app_sourcetype ] 
| stats list(datamodel) as datamodel, list(app) as app, list(app_sourcetype) as app_sourcetype by tag_name 
| search datamodel=* 
| stats values(datamodel) as datamodel, values(tag_name) as tags by app_sourcetype 
| eval tags=mvdedup(tags) 
| rex max_match=0 field=app_sourcetype "\"?(?<app>.+)__\"?" 
| rex max_match=0 field=app_sourcetype "__\"?(?<sourcetype>.+)\"?" 
| fields - app_sourcetype

purpose:

This search answers the questions which dashboards will my new add-on be used for in Enterprise Security.

requirements:

comments:

This is useful to see which app populates which datamodel in Enterprise Security or any other environment which datamodels.