Installation
Datajure requires Java 21+ and Clojure 1.12+.
With the Clojure CLI, add Datajure to deps.edn:
{:deps {com.github.clojure-finance/datajure {:mvn/version "2.7.12"}}}
With Leiningen, add it to :dependencies in project.clj:
[com.github.clojure-finance/datajure "2.7.12"]
The Clojars page always shows the latest version, and the changelog lists what changed in each release.
To try Datajure without creating a project, start a REPL with it on the classpath:
clj -Sdeps '{:deps {com.github.clojure-finance/datajure {:mvn/version "2.7.12"}}}'
A First Query
Datajure works on tech.ml.dataset datasets, which it brings in as a dependency. Create one from a map of columns:
(require '[datajure.core :refer [dt dt1 nrow asc desc]]
'[tech.v3.dataset :as ds])
(def prices
(ds/->dataset
{:date (map #(java.time.LocalDate/parse %)
["2024-01-02" "2024-01-03" "2024-01-04" "2024-01-05"
"2024-01-02" "2024-01-03" "2024-01-04" "2024-01-05"])
:sym ["AAPL" "AAPL" "AAPL" "AAPL" "MSFT" "MSFT" "MSFT" "MSFT"]
:price [100.0 102.0 99.0 104.0 200.0 198.0 202.0 207.0]
:volume [1200 1500 nil 1800 900 1100 1000 1300]}))
_unnamed [8 4]:
| :date | :sym | :price | :volume |
|------------|------|-------:|--------:|
| 2024-01-02 | AAPL | 100.0 | 1200 |
| 2024-01-03 | AAPL | 102.0 | 1500 |
| 2024-01-04 | AAPL | 99.0 | |
| 2024-01-05 | AAPL | 104.0 | 1800 |
| 2024-01-02 | MSFT | 200.0 | 900 |
| 2024-01-03 | MSFT | 198.0 | 1100 |
| 2024-01-04 | MSFT | 202.0 | 1000 |
| 2024-01-05 | MSFT | 207.0 | 1300 |
That is two stocks over four trading days. The nil is a missing value: there is no AAPL volume for 2024-01-04. The rest of this site uses this dataset.
Then query it with dt. Average price and total volume per symbol:
(dt prices
:by [:sym]
:agg {:n nrow
:avg-price #dt/e (mn :price)
:total-vol #dt/e (sm :volume)})
_unnamed [2 4]:
| :sym | :n | :avg-price | :total-vol |
|------|---:|-----------:|-----------:|
| AAPL | 4 | 101.25 | 4500 |
| MSFT | 4 | 201.75 | 4300 |
Reading the query part by part:
pricesis the dataset to query.:by [:sym]splits the rows into one group per symbol.:aggcollapses each group into one row. Its map names the new columns and says how to compute each one:nrowcounts the rows,mnis the mean andsmthe sum.#dt/emarks an expression. Inside it, bare keywords such as:pricerefer to columns.
The missing volume is skipped by the sum. The result is an ordinary dataset, so it can be passed to the next dt call, to tech.ml.dataset, or to tablecloth.
Other keywords filter and sort. The two highest prices with a volume above 1000:
(dt prices
:where #dt/e (> :volume 1000)
:order-by [(desc :price)]
:take 2)
_unnamed [2 4]:
| :date | :sym | :price | :volume |
|------------|------|-------:|--------:|
| 2024-01-05 | MSFT | 207.0 | 1300 |
| 2024-01-03 | MSFT | 198.0 | 1100 |
The row with the missing volume is dropped by :where, because a comparison with a missing value is false. The docs explain every keyword.
The first time a query does something worth knowing about, such as aggregating over the whole dataset, Datajure prints a one-time [datajure] NOTE: line explaining it.
Reading Files
datajure.io reads and writes files based on their extension:
(require '[datajure.io :as dio])
(def ds (dio/read "data.csv")) ;; also .tsv, .json, .jsonl, .parquet, .arrow, .xlsx, gzipped variants
(dio/write ds "output.csv")
Column names are read as keywords. Parquet, Arrow and Excel need the optional tech.ml.dataset libraries for those formats; see File I/O in the README.
REPL and Notebooks
With nREPL, the middleware datajure.nrepl/wrap-dt binds datajure.core/*dt* to the last dataset a form returned, similar to *1. Enable it in .nrepl.edn:
{:middleware [datajure.nrepl/wrap-dt]}
For notebooks, datajure.clay and datajure.clerk render datasets, #dt/e expressions and describe output. Call install! from the one you use (with Clay or Clerk on the classpath):
(require '[datajure.clay :as dc])
(dc/install!)