one flow.
one request.
one verdict.

An intrusion detection system that shows TypeSafe's Jev a few flows it has already seen, asks two typed questions about a new one and gets a verdict back, with no text to parse.

Tested on CIC-IDS2017, a public benchmark of network attacks, against a Random Forest, a kNN and an Isolation Forest. The 2,000 test flows are judged in the order they ended, and the true label of each one comes back D flows later, with D from 1 to 200.

With the label one flow late, the Random Forest scores a slightly higher F1, 0.988 against 0.984, a difference the paired test does not confirm. From D = 25 on, each of three runs of Jev IDS scores a higher F1 than every classical detector, and the lead grows with D.

input
flow        one CICFlowMeter record from CIC-IDS2017, 83 columns, compact
examples    chosen by the Selector, ceil(k) per category

request: two typed questions to jev
is_attack   noul     Is the flow an intrusion attempt?
category    choice   Which category does it belong to?

output
is_attack   → 0.80
category    → DoS, confidence 0.89
verdict     → attack   p_attack ≥ 0.58, the cut taken before the test

the problem

A flow is the summary of one conversation between two computers. It records how many packets and bytes went each way, how long the conversation lasted, which port it used and dozens of other measures. In CIC-IDS2017 each flow has 83 columns of measures and a label that says whether it is normal traffic or which attack it belongs to.

An intrusion detection system (IDS) receives each flow and decides whether it is benign traffic or part of an attack. When it decides attack, it raises an alarm. Two facts make the task hard.

The first is that attacks appear that the system has never seen. A detector that learned only from older examples tends to take such an attack for normal traffic.

zero-day attack. In network security, an attack the defenders do not yet know when it happens, so they had zero days to prepare. Here, the zero-day attacks are the three attack names of the test that never appear in the system's starting memory. Jev IDS meets them for the first time during the test, with no example of them.

The second is that the right answer arrives late. A security analyst confirms that a flow was an attack only some time later, when the damage shows or someone investigates. Until then the system keeps judging new flows without knowing whether it got the earlier ones right. This delay is called D and is counted in flows. With D = 1 the label of a flow comes back before the next flow is judged. With D = 200 it comes back only after 200 more flows.

how it works

Jev is an AI model from TypeSafe, of a kind called a System One Model (SOM). It reads a state, a block of information, and answers typed questions about it with probabilities, so there is no generated text to parse. Jev IDS puts the flow to judge into the state, next to the instructions, the column names, a description of each category and a few flows whose category is already known, the examples. Then it asks two questions.

  1. input

    one flow

    The record of a single connection, 83 columns, plus the examples the Selector chose.

  2. jev

    one request

    State: the instructions, the categories, the examples and the flow. Questions: is_attack and category.

  3. output

    one verdict

    attack · p_attack 0.80
    DoS · confidence 0.89

is_attack comes back as a probability from 0 to 1, called p_attack, and Jev IDS raises the alarm when it is at least 0.58. category comes back as one of six categories with a confidence, which says how concentrated Jev's probabilities over the six are. The values above are a real answer, for the first flow of the zero-day attack DoS Slowhttptest with D = 200.

the loop

Around that request runs a loop over the stream of flows. The top row of the diagram is one step, and the bottom row is what Jev IDS adds around Jev. The dashed line is the label of each judged flow, which comes back D flows later to the memory and to the KController.

The flow under test goes to jev, whose answer becomes the verdict. Below, the memory feeds the KController and the Selector, and the Selector sends the examples up to jev. A dashed line carries each label back D flows later to the memory and the KController. flow under test jev verdict memory KController Selector examples label back at t + D

The memory holds the labeled flows a step may draw examples from. These are the flows of the pool, prepared before the test, that ended before the flow under test, and every judged flow whose label has come back.

  1. The flow arrives. CIC-IDS2017 was recorded by a tool that writes a line when each connection ends, so a real IDS receives a flow only once it has ended. Jev IDS sorts the test flows by the moment each one ended and judges them one at a time in that order, so each step uses only information that already existed.
  2. The KController decides how many examples. It keeps one number k for each category, and Jev receives k examples of each category, rounded up. Every k starts at 1.
  3. The Selector decides which examples. Within each category it prefers the flows most similar to the flow under test, the most recent ones and the ones Jev was unsure about, and passes over near-duplicates so the examples show varied cases. Last, it adds the benign flow nearest to the flow under test, so Jev always sees the normal traffic closest to it.
  4. The compact record shortens the text. Each decimal value reaches Jev with 4 significant digits, so 25160.418 becomes 25160, and every column is kept. Jev charges for the tokens it reads, and this saves 21% to 25% of them with no loss in F1.
  5. Jev answers and Jev IDS decides. The alarm goes off when p_attack is at least 0.58. That cut was taken before the test, on a separate stream of 300 flows, and the test flows never tuned it.
  6. The label comes back D flows later. The judged flow joins the memory with its true category. When Jev named the wrong category, the k of the true category doubles, up to 8. When Jev was right, that k falls by 1/k, down to 1 at the least. Every label comes back, missed attacks included.

The KController's rule mirrors the one TCP uses against congestion on the internet (AIMD), which grows the sending window slowly and halves it on a loss. In an IDS the costly mistake is showing too few examples, so the KController grows k fast after an error and shrinks it slowly after a success.

Everything Jev is told lives in one versioned file, prompts/cicids2017/jev.json. Python adds only the flow and the examples, and the file's sha256 rides with every prediction, so any verdict traces back to the exact prompt. Examples are labeled by category only, and attack names never reach a model.

how it was tested

The test uses the relabelled CIC-IDS2017 of Liu et al. (IEEE CNS 2022), public traffic recorded on a lab network over five weekdays in July 2017, with attacks run by researchers. The flows of each label on each day are sorted by time and split in two halves. The earlier half may become examples, and the later half may be judged. The pool, the memory at the start of the test, is a sample of 88,066 flows of the earlier half. The 2,000 test flows were drawn from the later half, 1,020 benign and 980 attacks.

the six categories of the test · 2,000 flows
category what the traffic is attack names, with their flows
BENIGN normal traffic 1,020 flows
DoS an attempt to bring a server down with too many requests DoS Slowloris 101, DoS Slowhttptest 117 (zero-day), DoS Hulk 114, DoS GoldenEye 118
DDoS the same attack, from many machines at once DDoS 87
PortScan a sweep of a machine's ports in search of open services Portscan 106, Infiltration Portscan 89 (zero-day)
BruteForce guessing passwords by trial and error FTP-Patator 114, SSH-Patator 87 (zero-day)
Botnet infected machines that obey a remote controller Botnet 47

The three zero-day attack names never appear in the pool, so every detector meets them first during the test. SSH-Patator guesses the passwords of remote logins, DoS Slowhttptest brings a web server down with very slow requests, and Infiltration Portscan is a port sweep run from inside the network by a machine already broken into. Their categories stay in the pool through the other attack names. Together they hold 293 flows, and those flows measure the reaction to an unknown attack.

Each attack name arrives in one burst, a stretch of the stream where its flows come close together. The zero-day bursts span 116 to 171 flows. D = 1, 25, 50 and 100 are shorter than the shortest burst, so some labels of a zero-day attack come back while it still lasts. D = 200 is longer than the longest, so each detector has to recognise the attack without having seen it.

Jev IDS is compared with three classical detectors.

The three are refitted at every step on the same memory as Jev IDS, so at each judgement they know exactly what Jev IDS knows. A model trained once before the test would miss the labels that come back during it, and one trained on all the data would see flows from the future. They raise the alarm at p_attack ≥ 0.5, their usual cut.

Jev can answer slightly differently when it gets the same question twice, so Jev IDS ran three times at each D. The classical detectors use fixed seeds and give the same answers every time, so each ran once.

what the numbers mean

Every judged flow falls into one of four cases. An attack with an alarm is a hit. An attack without an alarm is a missed attack. A benign flow with an alarm is a false alarm. A benign flow without an alarm is a correct silence.

evidence

The paper-v2 split holds 2,000 flows of CIC-IDS2017 in the order they ended, drawn from rows no earlier split held and read once, in the runs of 2026-11-02. Jev IDS is the mean of its three runs and its false alarms the range over them. Each classical detector ran once.

paper-v2 split of CIC-IDS2017 · by delay D
D f1 zero-day recall false alarms / 1,020
jev ids rf knn if jev ids rf jev ids rf
1 0.984 0.988 0.983 0.495 0.953 0.945 1 to 2 1
25 0.975 0.961 0.951 0.494 0.883 0.799 2 0
50 0.969 0.947 0.934 0.494 0.861 0.713 3 to 6 0
100 0.954 0.906 0.898 0.495 0.721 0.471 2 to 4 0
200 0.923 0.873 0.869 0.495 0.540 0.294 3 0

With D = 1, Jev IDS and the Random Forest are level, because both learn almost at once from labels that come back one flow later. As D grows, the Random Forest falls fast, above all on zero-day attacks. Jev IDS falls more slowly, because Jev can judge an attack it has never seen from examples of the known ones. The Random Forest raises almost no false alarm, so the flows it loses are attacks it lets through. At D = 200 it misses all 87 flows of SSH-Patator and all 89 of Infiltration Portscan, where the first run of Jev IDS catches 62 and 28. The Isolation Forest never uses the returning labels, finds about 35% of all attacks at every D and raises 75 to 77 false alarms.

F1 and zero-day recall by delay D. Jev IDS stays above the Random Forest and the kNN from D = 25 on, and its lead grows with D, to 0.923 against 0.873 in F1 and 0.540 against 0.294 in zero-day recall at D = 200. The Isolation Forest stays flat, at 0.495 F1 and 0.328 zero-day recall. (a) F1, all 2,000 flows (b) zero-day recall, 293 flows 0.85 0.90 0.95 1.00 1 25 50 100 200 D, flows until a label comes back 0.00 0.25 0.50 0.75 1.00 1 25 50 100 200 D, flows until a label comes back jev 0.923 rf 0.873 knn 0.869 if 0.495, below the scale jev 0.540 rf 0.294 knn 0.287 if 0.328

Panel (a) shows F1 and panel (b) zero-day recall, both against the delay D, and higher is better in both. The delays are evenly spaced, so the step from 100 to 200 takes the same width as the step from 1 to 25.

Flow by flow, Jev IDS beats the Random Forest on zero-day attacks in every run from D = 25 on (adjusted p ≤ 0.042) and on all flows in every run from D = 50 on (adjusted p ≤ 0.0014). At D = 25 the win on all flows holds in two runs of three. At D = 1 no run differs from the forest on all flows, and in one run the forest is better on known attacks. Longer delays add nothing past D = 200, because the gap stops growing once the delay outlasts the bursts.

what each part adds

Each part of Jev IDS was removed or swapped in turn, one run each at D = 1, 50 and 200, and compared with the full system.

ablations · one run each · F1 and false alarms at D = 1 / 50 / 200
line what changes f1 false alarms
full system nothing 0.985 / 0.970 / 0.925 2 / 5 / 3
no selector examples drawn at random 0.925 / 0.935 / 0.886 69 / 65 / 58
fixed k = 1 no KController 0.978 / 0.962 / 0.916 3 / 3 / 4
fixed k = 4 no KController, 4 per category 0.990 / 0.978 / 0.928 6 / 6 / 8
full record every decimal kept 0.982 / 0.972 / 0.923 2 / 3 / 3
random, k = 4 no Selector, no KController 0.957 / 0.955 / 0.927 75 / 73 / 69
no nearest benign no benign flow added 0.979 / 0.971 / 0.929 27 / 29 / 22

The Selector matters most. Without it, Jev IDS loses on all flows at every D and raises 58 to 69 false alarms. The KController buys 0.7 to 0.8 points of F1 over one example per category for 5% to 10% more tokens, and four examples per category tie with the full system for twice the tokens. The nearest benign flow cuts the false alarms from 22 to 29 down to 2 to 5. The kNN, which reads the nearest flows without Jev, ties with Jev IDS at D = 1 and loses from D = 25 on, in the paired test of the first run.

limits worth knowing

earlier benchmark: nsl-kdd

Before the stream, Jev IDS was measured on NSL-KDD as a k-shot detector, with examples drawn at random and no feedback, against Gemini 3.6 Flash, a Random Forest and an Isolation Forest. At k = 1 it was 5.4 times faster and 22 times cheaper than the LLM, as precise (0.953 against 0.942) and lower on F1 (0.856 against 0.880). The tables are in docs/results.md.

try it

git clone https://github.com/jev-sec/jev-ids.git
cd jev-ids
uv sync
.env holds TYPESAFE_API_KEY for Jev.

Download CICIDS2017_improved.zip from the relabelled version of Liu et al. into data/raw/cicids2017/ and prepare it once.

uv run python -m scripts.prepare_cicids2017
uv run jev-ids run --dataset data/cicids2017/dataset.json --detector jev --split smoke --delay 1 --k 1 --control aimd --selection score+nearest --seeds 0

The smoke split is five flows. The run writes results/<timestamp>-cicids2017-jev-smoke/ with config.json, one JSON row per flow in predictions.jsonl and the raw answers in responses.jsonl. The README shows how to run the classical detectors and reproduce every number on this page.

bring your own flows

Jev IDS is an independent research prototype. Its numbers come from two benchmarks, CIC-IDS2017 and NSL-KDD. It can still sit inside a commercial solution, and this is how.

where jev fits

A commercial IDS already has sensors, a flow exporter and a signature engine feeding a SIEM. Jev IDS sits beside that pipeline, off the packet path, and judges one flow record at a time.

Sensor, exporter, signatures, alerts and SIEM in one line, and a side channel that leaves the exporter, goes through a queue to jev_ids and returns into the SIEM. sensor exporter signatures alerts SIEM queue jev_ids side channel

A quarter of a second per verdict and a rate-limited API make this an asynchronous side channel, fed from the exporter (NetFlow, IPFIX, Zeek conn.log) through a queue, never an inline filter.

six steps

  1. Get a key. Jev is served by TypeSafe. Set TYPESAFE_API_KEY in .env. Pricing and terms are TypeSafe's.
  2. Describe your flows. Write a dataset card like data/cicids2017/dataset.json, with the columns your flow exporter emits in order, which of them are symbolic, your categories and which one is benign.
  3. Write the request. Copy prompts/cicids2017/jev.json, replace columns and the category descriptions with yours, and keep the two questions.
  4. Seed the memory. One labeled flow per category from your own network is enough to start, because the KController begins at k = 1 and raises k only for the categories Jev gets wrong.
  5. Call the detector. JevDetector(load_prompt(path)).predict(flow, examples) returns p_attack, category_pred and confidence for one flow in about a quarter of a second, and stream.py shows how the memory, the Selector and the KController choose examples at each step. Route p_attack to your alerting with the cut your alarm budget allows. 0.58 was the benchmark's choice, not a rule.
  6. Measure before you trust. Run jev-ids run --delay and jev-ids metrics on a labeled split of your own flows. The numbers above are CIC-IDS2017's, not yours.

Only flow features leave your network, never payloads, but they do leave it. Every request goes to TypeSafe's API.

the team