LogLLM: Log-based Anomaly Detection Using Large Language Models (system log anomaly detection)

The final project of deep learning and practice (summer 2020) in NCTU.

A Python-based Intrusion Detection System (IDS) using Scapy to monitor network traffic for suspicious activity and potential threats. Detect anomalies, block malicious IPs, and log security incidents. 🛡️

A modular, machine learning-based system for detecting cybersecurity threats in real-time using log data, anomaly detection, and alerting mechanisms.

[code] "Real-Time Evasion Attacks against Deep Learning-Based Anomaly Detection from Distributed System Logs" by J. Dinal Herath, Ping Yang and Guanhua Yan. In: Proceedings of The 11th ACM Conference on Data and Application Security and Privacy (CODASPY) (2021).

This project focuses on enhancing cyclist safety using an ESP32-based IoT system that detects falls and abnormal cycling behavior in real time. By integrating ML/DL algorithms and anomaly detection on an ESP32 microcontroller, the system provides real-time alerts to emergency contacts and logs incident data on a cloud-based web platform.

Digital assaults assurance or discovery is by and by among the most testing research subjects of data security, while dramatically expanding in number of close to nothing, remote based associated gadgets ready to send individual data to the Web it is sitting idle yet causing the fight between the included individuals. Thus, this assurance gets significant with ordinary Internet of Things arrangement, as it oftentimes incorporates numerous IOT based information assets speaking with actual world inside the different application spaces, similar to horticulture, medical care, home computerization, and so on Lamentably, contemporary IoT based gadgets frequently offer an extremely restricted security determinations, exposing themselves to always new and more confounded assaults and furthermore hindering anticipated worldwide selection of the IoT innovations, also the great many the IoT gadgets previously delivered with no equipment security uphold. In this unique circumstance, it is fundamental to improve devices which can identify such digital assaults Interruption location is the way toward observing the functions happening in a PC framework or network and investigating them for indications of interruption. It expects to ensure the privacy, honesty, and accessibility of basic arranged data frameworks. Interruption location framework (IDS) is a framework that assembles and investigates data from different regions inside a PC or an organization to distinguish assaults made against these parts. The IDS utilizes various conventional strategies for checking the misuses of weaknesses. Present day, airplanes are made sure about by solid and wellbeing properties, prepared administrators, measure based safety efforts. Be that as it may, considering late development in the inflight administration towards the expanded network, the asset sharing and progressed amusement functionalities, along with increment of dangers focusing on installed frameworks, the possible malignant alteration of an airplane framework must be truly considered for future frameworks. In this specific circumstance, numerous arrangements can be produced for airplane security. Specifically, Host based Intrusion Detection Systems are appropriate to manage the focused on dangers like an insider-assault Intrusion detection systems are almost absolutely necessary in all types of networks to provide protection from intruders. Intrusion detection systems (IDS) have to process a lot of packets to detect any intrusion which causes a delay in detection and mitigation. A host-based IDS with rule structure generation and pattern matching algorithm sets the rule structure for the unknown attack by using association rule mining in the map reduce framework. It occurs in two different stages. An intellectual method is used to generate an efficacious rule in the first stage and a pattern matching algorithm is brute forced in the second stage of this proposed framework. Log reviewing and auditing is required to find any malicious activity. Windows is the most popular operating system in the world for personal computing needs. So, there are a large number of attacks happening every day on these systems and the built-in signaturebased detection methods are not suitable for detection of zero-day and stealth attacks. Intrusion based system based on anomaly Unfortunately, a comprehensive dataset that can identify surface operations and attacks are not available. To solve this, we are going to use Australian Defense Force Academy Windows Data Set with a Stealth Attacks Addendum (ADFA-WD: SAA). To make use of this dataset a highly intelligent host based intrusion detection system is required

A log-based anomaly detection system

Repo for IPDPS'23 Paper: "Drill: Log-based Anomaly Detection for Large-scale Storage Systems Using Source Code Analysis."

AI-based intrusion detection system that analyzes network traffic or system logs to detect potential security threats and anomalous activities. Utilize machine learning algorithms, such as anomaly detection or behavioral analysis, to identify patterns indicative of cyber attacks.

Cyber-security is concerned with protecting information, a vital asset in today’s world. The volume of data that is generated and can be usefully analysed is such that cyber-security can only be effectively implemented with the aid of software support. Data must be analysed by software tools providing support for security analysts. Often event data generated by computer systems is sequential, that is, not only are the type of the events relevant, but the sequence in which events occur is also relevant. Examples of this include many log files and system call or software library call sequences. This research aims to provide the basis to build an Anomaly Detection based Host Intrusion Detection System (HIDS) that makes decisions based on sequential traces of operating system calls.

Implementation of a Transformer based approach to anomaly-detection in system logs

A Cloud Intrusion Detection System (IDS) that leverages AWS CloudTrail logs to monitor and detect suspicious activities in a cloud environment. The system implements both rule-based and anomaly-based detection techniques to identify potential security threats, such as unauthorized public access to S3 buckets and unusual API call behaviors.

AnomLogBERT is a deep learning-based anomaly detection framework for system logs using BERT embeddings and neural network classifiers. It supports both Blue Gene/L (BGL) and Linux logs, enabling high-accuracy detection of abnormal events without manual log parsing.

R2Log: Robust and Real-time Anomaly Detection Based on Complete Logs for Large-scale Systems

This project provides an advanced anomaly detection system for Ceph distributed storage logs using LogBERT, a transformer-based language model fine-tuned for structured log analysis.

LogGuard is a Python-based system for real-time log monitoring and analysis. It features synthetic log generation, machine learning-driven anomaly detection, efficient data compression, and a fast, searchable index for forensic analysis

User Behavior Analytics (UBA) for Insider Threat Detection. A Python-based system that uses machine learning (Isolation Forest) to analyze user behavior logs, detect anomalies, and identify potential insider threats within an organization's network. The project includes data preprocessing, model training, and a Flask API for anomaly detection.

An Intrusion Detection System (IDS) is a security application that monitors a system or network for unauthorized access, anomalies, or suspicious activities. The IDS developed using Django is a web-based solution that detects intrusions, logs incidents, and notifies users in real-time.

Anomaly Detection with Text Based System logs

Log based anomaly detection and fault localization system.

CLI-based log anomaly detection system for Linux servers

AI-based anomaly detection system for telecom RAN log analysis with streaming and window-based ML detection.

Investigation of anomaly detection methods in event logs and event models based on rule systems

Efficient LLM-based log analysis system for multi-cloud environments with real-time monitoring and anomaly detection.

Raspberry Pi-based real-time energy monitoring system with current/voltage sensors, logging, and signal processing for anomaly detection.

A machine learning-based intrusion detection system to detect network anomalies in real time. Includes preprocessing, model evaluation, and alert logging.

Real-time hiring log ingestion and forecasting system using PySpark + AWS. Simulates candidate application logs, performs anomaly detection, and builds ML-based demand forecasts to optimize workforce planning.

A composable, skill-based agent system that transforms raw logs into actionable insights. Features: log discovery, parsing, aggregation, anomaly detection, summary generation, and actionable recommendations. Built for SREs and DevOps teams.

Real-time network intrusion detection system using Zeek flow logs and machine learning (IsolationForest). Detects threats with both signature-based and anomaly-based techniques trained on the CSE-CIC-IDS2018 dataset.