In this blog we will talk about the processes and the options we have to collect the GCP events and we will see how to collect those in Splunk. In addition, we will even add integration with Cribl, as an optional step, in order to facilitate and optimize the process of information ingestion. After synthesizing all of this great information, you will have a great understanding of the available options to take, depending on the conditions of the project or team in which you work.
https://discoveredintelligence.com/wp-content/uploads/2023/08/gcp_splunk_spotlight.png400600Discovered Intelligencehttps://discoveredintelligence.com/wp-content/uploads/2013/12/DI-Logo1-300x137.pngDiscovered Intelligence2023-08-09 16:34:142023-08-29 01:04:31Building a Unified View: Integrating Google Cloud Platform Events with Splunk
Are you tired of hardcoding email addresses into your searches and alerts? Do you want a more dynamic way to send search results to individuals based on the data within your search results? Look no further than SendResults, a powerful Splunk command and alert action developed by Discovered Intelligence.
https://discoveredintelligence.com/wp-content/uploads/2023/08/sendresults_command.png400600Goutham Vinaya Babuhttps://discoveredintelligence.com/wp-content/uploads/2013/12/DI-Logo1-300x137.pngGoutham Vinaya Babu2023-08-09 14:55:212023-08-09 14:55:22Discover the Power of SendResults: A Life-Changing Splunk Command and Alert Action
If you haven’t heard of ChatGPT yet, you likely have blocked notifications on social networks like Linkedin, Twitter or Reddit, as everyone is talking about the benefits (and concerns) of artificial intelligence. However, it’s ChatGPT who gets the lion’s share of the limelight in this story.
https://discoveredintelligence.com/wp-content/uploads/2023/05/chatgpt_help.png353685Carlos Moreno Buitragohttps://discoveredintelligence.com/wp-content/uploads/2013/12/DI-Logo1-300x137.pngCarlos Moreno Buitrago2023-05-31 23:53:072023-06-01 13:53:05ChatGPT and SPL: A Dynamic Duo for Learning Splunk’s Query Language
Organizations of all sizes are building / migrating / refactoring their software to be cloud-native. Applications are broken down into microservices and deployed as containers. Consequently there has been a seismic shift in the complexity of application components thanks to the intricate network of microservices calling each other. The traditional sense of “monitoring” them no longer makes sense, especially because containers are ephemeral in nature and are treated as cattle, instead of as pets.
https://discoveredintelligence.com/wp-content/uploads/2023/05/k8s_otel_collection.jpg440800Thulasi Krishnanhttps://discoveredintelligence.com/wp-content/uploads/2013/12/DI-Logo1-300x137.pngThulasi Krishnan2023-05-12 14:25:572023-05-12 14:25:58Wiring up the Splunk OpenTelemetry Collector for Kubernetes
Workload management is a powerful Splunk Enterprise feature for users to delegate CPU and memory resources to various Splunk workloads, based on their preferences. As Splunk continues to develop new attributes for the defining of rules, the number of Splunk users who are enabling workload management in their environment is gradually increasing.
The quote from Check Point Research above illustrates where the future trend of cybersecurity is headed and the challenges that organizations must face. However, anticipating and preparing the system defenses to evade and mitigate these attacks is not an easy task. From defining response and incident strategies to preparing work teams and configuring monitoring systems, it can all be a challenge.
Your core business is not to detect and mitigate security attacks, but is this essential to the achievement of your objectives? Have you ever wondered how you can simulate attacks and detections within a controlled environment to validate the configuration of your detection systems without spending part of your annual security budget? Read on and discover Splunk Attack Range.
What is Splunk Attack Range?
Splunk Attack Range is a tool developed by Splunk Threat Research Team (STRT) to simulate cyber attacks in a controlled environment for the purpose of improving an organization’s security posture. It allows security teams to test and validate their detection and response capabilities against a wide range of attack scenarios and techniques, such as phishing, malware infections, lateral movement, and data exfiltration.
Splunk Attack Range is designed to work with Splunk Enterprise Security, which is a security information and event management (SIEM) solution, and includes pre-built attack scenarios that are aligned with the MITRE ATT&CK framework, these ones can be customized to simulate the specific threats and vulnerabilities that are relevant to an organization’s environment.
Where can I get Attack Range?
The STRT and the Splunk community are maintaining the project in GitHub.
Is Splunk Attack Range Easy to Deploy?
Yes, it is really straightforward! You can deploy it locally (if you have a powerful machine), on Azure or on AWS. Internally, we use our AWS environment and with a few simple steps, in a matter of minutes, terraform and ansible automatically deploy a complete test lab to validate our customers’ security configurations and optimize the security posture with Splunk’s real-time monitoring. This process allows for a proactive approach to managing security postures with Splunk and saves a lot of time for your Blue Team.
…and now?
Have fun! By merging our Splunk expertise and using these kinds of automation tools, we have been able to speed up our internal testing processes, stay agile and secure with Splunk’s security posture management tool, and transfer this knowledge and configurations on to our customers’ cybersecurity teams.
We strongly encourage you to try this tool. Check out an overview of v1.0, v2.0 and v3.0 in the Splunk blog.
https://discoveredintelligence.com/wp-content/uploads/2023/03/splunk-attack-range-logo-e1678466676167.png693696Carlos Moreno Buitragohttps://discoveredintelligence.com/wp-content/uploads/2013/12/DI-Logo1-300x137.pngCarlos Moreno Buitrago2023-03-14 15:17:262023-03-14 15:17:29Save Time and Improve your Security Posture with Splunk Attack Range
Splunk Cloud Admins rejoice! The Splunk Cloud ACS Command Line Interface is here! Originally, the Splunk Cloud Admin Config Service (ACS) was released in January 2021 to provide various self-service features for Splunk Cloud Admins. It was released as an API-based service that can be used for configuring IP allow lists, configuring outbound ports, managing HEC tokens, and many more which are all detailed in the Splunk ACS Documentation.
To our excitement Splunk has recently released a CLI version of ACS. The ACS CLI is much easier to use and less error-prone compared to the complex curl commands or Postman setup one has had to deal with to-date. One big advantage we see with the ACS CLI is how it can be used in scripted approach or within a deployment CI/CD pipeline to handle application management and index management.
We would recommend that you first refer to the ACS Compatibility Matrix to understand what features are available to the Classic and Victoria experience Splunk Cloud platforms.
ACS CLI Setup Requirements
Before you get started with the ACS CLI there are a few requirements to be aware of:
You must have the sc_admin role to be able to leverage the ACS CLI.
You must be running a Mac or Linux operating systems. However, if you are a Windows user you can use the Windows Subsystem for Linux (WSL), or any Linux VM running on Windows, to install and use the ACS CLI.
The Splunk Cloud version you are interacting with must be above 8.2.2109 to use the ACS CLI. To use Application Management functions, your Splunk Cloud version must be 8.2.2112 or greater.
Please refer to the Splunk ACS CLI documentation for further information regarding the requirements and the setup process.
ACS CLI Logging
At the time of authoring this blog, logging and auditing of interactions through the Splunk Cloud ACS is not readily available to customers. However, when using the ACS CLI it will create a local log on the system where it is being used. It is recommended that any administrators given access to work with the ACS CLI have the log file listed below collected and forwarded to the their Splunk Cloud stack. This log file can be collected using the Splunk Universal Forwarder, or other mechanism, to create an audit trail of activities.
Linux: $HOME/.acs/logs/acs.log
Mac: $HOME/Library/Logs/acs/acs.log
The acs.log allows an administrator to understand what operations were run, request IDs, status codes and much more. We will keep an eye out for Splunk adding to the logging and auditing functionality not just in the ACS CLI but ACS as a whole and provide a future blog post on the topic when available.
Interacting With The ACS CLI
Below are examples of common interactions an administrator might have with Splunk Cloud now done by leveraging the Splunk Cloud ACS CLI. There are many more self-service features supported by the ACS CLI, details of the supported features and CLI operations are available in the Splunk Cloud ACS CLI documentation.
Application Management
One of the most exciting features of the ACS CLI is the ability to control all aspects of application management. That means, using the ACS CLI you can install both private applications and Splunkbase applications.
The command is easy to understand and straightforward, for both private and Splunkbase applications it supports commands to install, uninstall, describe applications within your environment as well as a list command to return a complete list of all installed applications, with their configurations. Specific to only Splunkbase applications there is an update command which allows you to, you guessed it, update the application to the latest version published and available.
For both private and Splunkbase apps, when running a command it will prompt you to enter your splunk.com credentials. You can pass –username –password parameters along with the command to avoid prompting for credentials. For private apps these credentials will be used to authenticate to AppInspect for application vetting.
Application Management: Installing a Private App
Let’s look at how we use the ACS CLI to install a private application. The following command will install a private app named company_test_app:
acs apps install private --acs-legal-ack Y --app-package /tmp/company_test_app.tgz
Now when a private app is installed using the ACS CLI it will automatically be submitted to AppInspect for vetting. A successful execution of the command will result in the following response, which you will note includes the AppInspect summary:
Submitted app for inspection (requestId='*******-****-****-****-************') Waiting for inspection to finish... processing.. success Vetting completed, summary: { "error": 0, "failure": 0, "skipped": 0, "manual_check": 0, "not_applicable": 56, "warning": 1, "success": 161 } Vetting successful Installing the app... { "appID": "company_test_app", "label": "Company Test App", "name": "company_test_app", "status": "installed", "version": "1.0.0" }
Application Management: Installing a Splunkbase Application
Let’s now look at an example of installing a Splunkbase application by running a command to install the Config Quest application:
The licensing URL passed as a parameter in the command above can be found in the application details on Splunkbase. Additionally, by running a curl command the licensing URL can be retrieved from the Splunkbase API:
Index management using the ACS CLI supports a wide range of functionality. The supported commands allow you to create, update, delete and describe an index within your environment as well as a list command to return a list of all of the existing indexes, with their configurations.
Let’s now look at how we run one of these commands by running a command that creates a metrics index with 90 days searchable retention period. Note that ACS supports creating either event or metrics index, however it does not yet support configuring DDAA or DDSS.
Managing HTTP Event Collector (HEC) token’s just got real easy. The ACS CLI supports commands to create, update, delete and describe a HEC token within your environment as well as a list command to return a list of all of the existing HEC token’s, with their configurations.
Let’s now look at how we run one of these commands by running a command to create a HEC token in Splunk Cloud quickly and easily:
acs hec-token create --name test_token --default-index main --default-source-type test
A successful execution of the command provides the token value in the JSON response:
Planning a sequel to the blog –Moving bits around: Deploying Splunk Apps with Github Actions – led me to an interesting experiment. What if we could manage and automate the deployment server the same way, without having to log on to the server at all. After all, the deployment server is just a bunch of app directories and a serverclass.conf file.
The Splunk Machine Learning Toolkit is packed with machine learning algorithms, new visualizations, web assistant and much more. This blog sheds light on some features and commands in Splunk Machine Learning Toolkit (MLTK) or Core Splunk Enterprise that are lesser known and will assist you in various steps of your model creation or development. With each new release of the Splunk or Splunk MLTK a catalog of new commands are available. I attempt to highlight commands that have helped in some data science or analytical use-cases in this blog.
https://discoveredintelligence.com/wp-content/uploads/2020/10/image-10.png2561892Discovered Intelligencehttps://discoveredintelligence.com/wp-content/uploads/2013/12/DI-Logo1-300x137.pngDiscovered Intelligence2020-11-12 20:24:382022-10-31 14:41:21Interesting Splunk MLTK Features for Machine Learning (ML) Development
Recently a customer was reviewing asset information in Aura Asset Intelligence, our premium application for Splunk, and some interesting data showed up. Users had mobile devices that had emoji’s in their name of their device.
It was a bit surprising at first as it’s not what you would normally expect in a corporate IT environment, but after thinking about it, it’s perfectly normal to see – especially with companies fully adopting BYOD programs these days.
If you weren’t already aware, Splunk can handle different character sets. You can work with non-ascii characters in various different ways – including emojis! From indexing data, searches, alerts, and dashboards. Once you get into the world of non-ascii, you are dealing with Unicode. Unicode is a complex topic. There are many different concepts and terminology to keep straight. But that’s not really the point of this blog 😉 . For more information on Unicode you can start here.
It certainly gets you thinking 🤔 , where could emojis be used in Splunk to inject a bit of fun. Why not give your searches and Splunk dashboards a little ❤️ ?
Errors single-value panel: index=main sourcetype=access_combined | stats count(eval(status >= 500)) as errors count as total | eval error_rate=round((errors/total)*100,1) | eval alt_status = if(error_rate >= 3, "😕","😄")| fields alt_status
Status Codes table panel: index=main sourcetype=access_combined | stats count by status | eval alt_status = case(status >= 500, "😠",status >=400, "😕", status >= 200, "😄", 1==1,"❓")
Or even using them in alerts (results will vary depending if the target of the alert can handle Unicode). Here’s an email example with the results embedded inline:
Maybe you can live on the wild side and even ask your developers to start using emoji’s in their logs….
Ok, that’s fun and all, but is there a practical use for emoji’s in Splunk? Sure! Why not give your dashboards some more visual eye candy when it comes to location data. You can easily create a lookup that maps Country name to their emoji flag.
Top Country single-value panel: index=main sourcetype="access_combined" | top limit=1 clientip | iplocation clientip | eval Country = if(Country=="", "Unknown", Country) | lookup emoji_flags name as Country OUTPUT emoji | fillnull value="❓" emoji | eval top_country= Country." ".emoji | fields top_country
Requests By Country table panel: index=main sourcetype="access_combined" | stats count by clientip | iplocation clientip | eval Country = if(Country=="", "Unknown", Country) | stats sum(count) as total by Country | lookup emoji_flags name as Country OUTPUT emoji | fillnull value="❓" emoji | sort - total
You can download the flag to emoji lookup CSV here to use in your own searches.
The possibilities are endless! So have some fun with emojis in your dashboards, lets just hope that at no point do your dashboards or data go to 💩 …
