join

It's a pretty popular search command and it is used in all sorts of situations. Below are some really cool searches that use join along with other search commands.

Chart HTTP Status Category % by URL (using join)

index=* sourcetype=access* status=* | rex field=bc_uri "/(?<route>[^/]*)/" | stats count as scount by route, status_type | join route [search index=* sourcetype=access* status=* | rex field=bc_uri "/(?<route>[^/]*)/" | stats count as ttl by route] | eval pct = round((scount / ttl), 2)."%" | xyseries route status_type pct

purpose:

requirements:

comments:

Search Golf - Episode 1

# source the events in chron order (so "start" is before "end")
index=cst sourcetype=mav-golf | reverse 
# add a line number / temp id to the events
| eval lc=1 | accum lc 
# extract a field to make it easier to deal with action
#  not really necessary in this example - could just search for "start" / "end"
| rex field=_raw "ID=\S\s(?<action>\S+)\s" | stats list(action) as action by ID, lc 
# find action=start for each identifier and join that back into each row
| join ID type=left [search index=cst sourcetype=mav-golf | reverse | eval lc=1 | accum lc | rex field=_raw "ID=\S\s(?<action>\S+)\s"  | search action=start | stats first(lc) as open by ID] 
# find action=end for each identifier and join that back into each row
| join ID type=left [search index=cst sourcetype=mav-golf | reverse | eval lc=1 | accum lc | rex field=_raw "ID=\S\s(?<action>\S+)\s"  | search action=end | stats last(lc) as close by ID] 
# lastly, test each event to see if it's own id is between the start and end.
#  if so - count it.
| eval sc = if(lc>open, if(lc<close, 1, 0), 0) 
# And then sum up those events which should be counted.
| stats sum(sc) as num_events by ID

purpose:

Find the number of events within a sequence of events based on a shared identifier. Keywords ("start" and "end") mark the beginning and end of the sequence. The search cannot use the transaction command.

requirements:

Data like the following: 01/01/2014 01:01:00.003 ID=a start blah blah 01/01/2014 01:01:01.003 ID=d more blah blah 01/01/2014 01:01:02.003 ID=a end blah blah 01/01/2014 01:01:03.003 ID=b start blah blah 01/01/2014 01:01:04.003 ID=c start blah blah 01/01/2014 01:01:05.003 ID=y more blah blah 01/01/2014 01:01:05.006 ID=c more blah blah 01/01/2014 01:01:05.033 ID=c more blah blah 01/01/2014 01:01:06.003 ID=c end blah blah 01/01/2014 01:01:06.033 ID=b more blah blah 01/01/2014 01:01:07.003 ID=b end blah blah 01/01/2014 01:01:08.004 ID=c more blah blah 01/01/2014 01:01:09.005 ID=b more blah blah

comments:

User Search Restrictions and Last Logins

| rest /services/authentication/users splunk_server=local
| table title realname email roles 
| mvexpand roles 
| join roles type=outer 
    [| rest /services/authorization/roles splunk_server=local
    | fields imported* title srch* 
    | fields - *Quota *TimeWin *capabilities 
    | rex mode=sed field=srchFilter "s/^\*$/true/" 
    | rex mode=sed field=imported_srchFilter "s/^\*$/true/" 
    | eval search_restrictions=if(imported_srchFilter="","( ".srchFilter." )",if(srchFilter="","( ".imported_srchFilter." )","( ".srchFilter." ) OR ( ".imported_srchFilter." ) ")) 
    | fields - srchIndexesDefault 
    | eval srchIndexesAllowed_new=mvjoin(srchIndexesAllowed," OR index=") 
    | eval index_restrictions="( index=".srchIndexesAllowed_new." )" 
    | table title search_restrictions index_restrictions 
    | rename title AS roles ] 
| stats values(roles) AS roles values(search_restrictions) AS search_restrictions values(index_restrictions) AS index_restrictions by title,realname,email 
| rex mode=sed field=search_restrictions "s/\( \)//g" 
| eval search_restrictions=mvjoin(search_restrictions," OR ") 
| eval index_restrictions=mvjoin(index_restrictions," OR ") 
| eval realname=upper(realname)
| rename title AS username
| eval restrictions=if(search_restrictions="( )",index_restrictions,search_restrictions." ".index_restrictions)
| table username email username roles restrictions
| search username=* 
| rex mode=sed field=restrictions "s/^ OR //g" 
| join type=outer username 
    [ search (index=_audit "login attempt info=succeeded) OR (index=_internal "GET /splunk/en-US/account/login") 
    | stats max(_time) AS "last_login" by user 
    | rename user AS username ] 
| eval days_ago=floor((now()-last_login)/86400) 
| eval last_login=strftime(last_login,"%Y-%m-%d %H:%M:%S") 
| fillnull last_login value="Never"

purpose:

requirements:

comments:

Shows a table showing the inherited search and index restrictions for each user, represented as an SPL string. In certain cases it may not be 100% accurate, but it covers most instances that I have run into. NOTE: does not account for `searchFilterSelecting = false` defined in authorize.conf