Datajure

Clojure · tech.ml.dataset · Apache 2.0

Ask questions of your data with one function.

Datajure gives Clojure a compact query language for tables. Filter rows, add columns, group, aggregate and run window functions with a single function, dt, and a few keywords. Every result is an ordinary dataset.

Get started Read the docs
Latest version on Clojars CI status cljdoc
REPL
(dt prices
    :where #dt/e (> :volume 1000)  ; busy days only
    :by    [:sym]                  ; one group per stock
    :agg   {:days      nrow        ; count the rows
            :avg-price #dt/e (mn :price)})
Result
:sym:days:avg-price
AAPL3102.0
MSFT2202.5

One query function

:where, :set, :agg, :by and a few more keywords cover filtering, new columns, grouping and sorting.

Time series and cross-sections

Lags, rolling windows and as-of joins down each series; ranks, z-scores, winsorizing and quantile buckets across each date. Panel data gets both.

Expressions are values

Write #dt/e (* :price :volume) once, store it in a var and reuse it across queries.

Safe with missing data

Comparisons with a missing value are false and aggregations skip it, so blank cells don’t derail a query.

Fast, not magic

Expressions compile to vectorized column operations on tech.ml.dataset. Datajure adds syntax, not a new engine.

Errors that help

A typo such as :prcie is caught before the query runs, with a suggestion: did you mean :price?

See It in Action

The examples use prices, a small table of daily prices for two stocks, defined in the quick start. Every output on this site comes from running the code.

Compute within Groups

Daily returns per stock, in date order. Every row is kept:

(dt prices
    :by [:sym]
    :within-order [(asc :date)]
    :set {:ret #dt/e (- (win/ratio :price) 1)})

_unnamed [8 5]:

|      :date | :sym | :price | :volume |        :ret |
|------------|------|-------:|--------:|------------:|
| 2024-01-02 | AAPL |  100.0 |    1200 |             |
| 2024-01-03 | AAPL |  102.0 |    1500 |  0.02000000 |
| 2024-01-04 | AAPL |   99.0 |         | -0.02941176 |
| 2024-01-05 | AAPL |  104.0 |    1800 |  0.05050505 |
| 2024-01-02 | MSFT |  200.0 |     900 |             |
| 2024-01-03 | MSFT |  198.0 |    1100 | -0.01000000 |
| 2024-01-04 | MSFT |  202.0 |    1000 |  0.02020202 |
| 2024-01-05 | MSFT |  207.0 |    1300 |  0.02475248 |

Chain Steps

Each dt call returns a dataset, so steps chain with ->. Returns, then their mean and volatility per stock, then a sort:

(-> prices
    (dt :by [:sym] :within-order [(asc :date)]
        :set {:ret #dt/e (- (win/ratio :price) 1)})
    (dt :by [:sym] :agg {:mean-ret #dt/e (mn :ret) :vol #dt/e (sd :ret)})
    (dt :order-by [(desc :mean-ret)]))

_unnamed [2 3]:

| :sym |  :mean-ret |       :vol |
|------|-----------:|-----------:|
| AAPL | 0.01369776 | 0.04032943 |
| MSFT | 0.01165150 | 0.01888828 |

Compare across Stocks

Group by date instead of by stock to work across the cross-section. Each return minus that day's average return:

(-> prices
    (dt :by [:sym] :within-order [(asc :date)]
        :set {:ret #dt/e (- (win/ratio :price) 1)})
    (dt :by [:date]
        :set {:excess #dt/e (stat/demean :ret)}
        :select [:date :sym :ret :excess]
        :order-by [(asc :date)]))

_unnamed [8 4]:

|      :date | :sym |        :ret |     :excess |
|------------|------|------------:|------------:|
| 2024-01-02 | AAPL |             |             |
| 2024-01-02 | MSFT |             |             |
| 2024-01-03 | AAPL |  0.02000000 |  0.01500000 |
| 2024-01-03 | MSFT | -0.01000000 | -0.01500000 |
| 2024-01-04 | AAPL | -0.02941176 | -0.02480689 |
| 2024-01-04 | MSFT |  0.02020202 |  0.02480689 |
| 2024-01-05 | AAPL |  0.05050505 |  0.01287629 |
| 2024-01-05 | MSFT |  0.02475248 | -0.01287629 |

Match by Time

An as-of join gives each trade the latest quote for its symbol at or before the trade's time:

(join trades quotes :on [:sym :time] :how :asof)

_unnamed [3 4]:

| :sym | :time | :qty |  :bid |
|------|------:|-----:|------:|
| AAPL |    10 |  100 |  99.5 |
| AAPL |    25 |   50 | 100.5 |
| MSFT |    30 |   75 | 201.0 |

The Core Idea

Two keywords decide what a query does to the rows. :set adds columns and keeps every row; :agg collapses rows into summaries. :by splits the rows into groups first. Together they give four operations:

Around that core, :where filters rows first, and :select, :order-by and :take shape the result. The docs explain each step.

Why Datajure

Datajure is a syntax layer, not an engine. It compiles #dt/e expressions to vectorized operations on tech.ml.dataset, so results are standard datasets that work with tablecloth, Clerk, Clay and the rest of the Scicloj ecosystem.

Its design borrows from whichever library got a given idea right:

  • R's data.table: DT[i, j, by] becomes (dt ds :where i :set-or-agg j :by by), without in-place changes.
  • Python's Polars: expressions are values you can store and combine, and comparisons with missing values are safe.
  • q/kdb+: time-series tools such as win/ratio, win/mavg, win/ema, win/fills and xbar, plus as-of and window joins.
  • Julia's DataFramesMeta.jl: #dt/e plays the role of @transform and @subset, using a Clojure reader tag.

Where to Go Next

Quick StartInstall Datajure and run your first query in a few minutes.DocsHow queries work: keywords, expressions, missing values, grouping and joins.ExamplesCommon tasks on one small dataset, each with its output.Time SeriesReturns, rolling windows, gaps, bars and as-of joins for panel data.From R & PythonThe same tasks in dplyr, data.table, pandas, Polars and Datajure.ReferenceThe README covers every option; API docs are on cljdoc.

Datajure v1

Datajure v2 (March 2026) is a rewrite. Datajure v1 (1.x) was a query syntax that routed to several backends: tech.ml.dataset, tablecloth, Clojask and Geni. Its code is on the v1 branch, its documentation is in this site's history, and posts from that time describe its design. Thanks to YANG Ming-Tian for the original v1 implementation.

Found a bug? Please open an issue. Datajure is distributed under the Apache License, Version 2.0.