We are currently trying to get our feet wet in managing our own parsers in Chronicle. We have started with Virtru Email Encryption logs which are ingested as JSON. I have been through the documentation quite a bit now but feel that a little bit of help would go a long way. I found another post stating that we need to import as a single line, which I believe we can do. Here are the JSON fields we are trying to target in the log in BOLD:
{ "id": "fa894a4a-9471-4cd1-a6d8-9ad9f26c6593", "object": { "type": "data_object", "id": "7ef24565-b549-4fab-8bef-7ea0402a5243", "name": "EHLERS.pdf", "attributes": { "attrs": [ { "key": "virtru:data:policy:type", "value": "file" }, { "key": "virtru:data:creator", "value": "cdward@example.org" }, { "key": "virtru:data:owner", "value": "cdward@example.org" },
], "dissem": [ "margaret.mann@example.org", "audra@example.org", "gwen@example.org" ]
"actor": { "id": "gwen@example.org", "attributes": { "attributes": [ {
}, "clientInfo": { "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36", "platform": "browser_extension_chrome:12.0.2", "requestIp": "192.168.5.123" }, "eventMetaData": { "auditDataType": "FILE.ACCESS_SUCCEEDED", "auditRecordId": "bc133671-e9a6-4313-8482-e2a5fe007fef", "auditRecordType": "contract-get", "errorDetails": null }, "timestamp": "2024-02-14T22:12:18.321Z" }
I have bolded all of the fields we want to map into UDM as our test parser build. We can add more later but this would be a great start.
This is how we believe the fields line up but again, we are new to this. If you have suggestions on UDM field changes, please let me know:
name > about.email
virtru:data:policy:type > event.idm.read_only_udm.metadata.description
virtru:data:owner > src.email
dissem > target.email (this is the email recipient)
actor:id > principal.user.user_display_name
requestIp > principal.ip
auditDateType: > security_result.action_details
timestamp > event_timestamp
Here are the fields to create in Chronicle:
event.idm.read_only_udm.metadata.vendor_name = Virtru
event.idm.read_only_udm.metadata.product_name = virtru encryption
event.idm.read_only_udm.metadata.event_type = Generic_Event
Thank you for your help.
Here is my first crack at the parser:
filter {
json {
source => "message"
array_function => "split_columns"
}
#TODO(add error handling in case of JSON extraction failure)
mutate {
replace => {
"event1.idm.read_only_udm.metadata.event_type" => "GENERIC_EVENT"
}
}
#TODO(evalute a more specific UDM event type)
mutate {
replace => {
"event.idm.read_only_udm.metadata.vendor_name" => "Virtru"
}
}
mutate {
replace => {
"event.idm.read_only_udm.metadata.product_name" => "virtru encryption"
}
}
mutate {
replace => {
"about.email.key" => "object_name"
"about.email.value.string_value => "%{object.name}"
}
}
mutate {
replace => {
"event.idm.read_only_udm.metadata.description.key" => "object_attributes_attrs.0_key"
"event.idm.read_only_udm.metadata.description.value.string_value => "%{object.object_attributes_attrs.1_value}"
}
}
mutate {
replace => {
"src.email.key" => "object_attributes_attrs.2_key"
"src.email.value.string_value => "%{object_attributes_attrs.2_value}"
}
}
mutate {
replace => {
"target.email.key" => "object_attributes_dissem"
"srctarget.email.value.string_value => "%{object_attributes_dissem.[0]}"
}
}
mutate {
replace => {
"principal.user.user_display_name.key" => "actor_id"
"principal.user.user_display_name.value.string_value => "%{actor.id[0]}"
}
}
mutate {
replace => {
"principal.ip.key" => "clientInfo_requestIp"
"principal.ip.value.string_value => "%{clientInfo_requestIp[0]}"
}
}
mutate {
replace => {
"security_result.action_details.key" => "eventMetaData_auditDataType"
"security_result.action_details.value.string_value => "%{eventMetaData_auditDataType[0]}"
}
}
mutate {
replace => {
"event_timestamp.key" => "timestamp"
"event_timestamp.value.string_value => "%{timestamp[0]}"
}
}
mutate {
merge => {
"@output" => "event1"
}
}
}
Try looking closer at these examples of the UDM model. A lot of your Field Names are out of place. The email values for one aren't going to work like that.
https://cloud.google.com/chronicle/docs/event-processing/udm-overview