How it works
dailies, from a messy CSV to an honest verdict

Plenty of people keep a daily log of migraines, gut trouble, sleep or mood, then still cannot say which factors matter. Diary apps mostly show a grid of pairwise correlations, one factor against the symptom at a time. That misses the common case where a bad day needs two things at once, like a short night and a skipped meal. And health data is exactly what people least want to hand to a server.

Dailies screens your own export in the tab. Nothing is uploaded, and the only storage is an optional copy in your own browser. It answers two questions in order: is there any signal at all, and if so, which factors carry it.

The CSV parser is hand written. It sniffs the delimiter (comma, semicolon, tab or pipe), honours a sep= line, handles quoted cells with doubled quotes and line breaks, and strips a byte order mark. Each column is then typed by its values: dates (ISO, day first or month first, or named months), yes or no, numbers (with units like 160 mg, comma decimals, or 7:30 as hours), lists split on a pipe as Daylio writes activities, small categories, and free text, which is left out. Rows that share a date are merged: the last non blank value wins, and lists are joined.

number      z = (x − mean) / sd           blank → 0, the average
yes / no    1 or 0, then z scored          blank → 0
category    one column per level, z scored blank → 0 on every level
blank flag  1 if the cell was blank        only when 6+ days are blank
yesterday   the same signals, shifted one calendar day

Z scoring puts every input on the same scale, which keeps one learning rate sensible and makes the credits comparable. A blank becomes the average plus, when blanks are common enough to learn from, a flag of its own, because not logging can itself mean something. A flag seen on three days is noise the network will happily memorise, so rarer blanks get no flag.

Days are split into five folds of consecutive days, not shuffled. Habits drift over months, and neighbouring days resemble each other, so a shuffled split lets the model peek at the answer. Each fold trains three models on the other four fifths and scores them on the held out stretch:

base rate   predict the training share of bad days, every day
logistic    one linear layer, each factor alone          [n, 1]
MLP         one tanh hidden layer, factors combined       [n, 8, 1]

gain    = mean over folds of (log-loss base − log-loss MLP)
spread  = sample standard deviation of those 5 differences
signal  ⇔ gain > spread and gain > 0.01

Log-loss is the score because accuracy flatters the base rate when bad days are rare: a log with 20% bad days gets 80% accuracy by always saying fine. If the MLP does not beat the base rate by more than its own fold to fold wobble, the verdict is no reliable signal yet. If it does but the rarer side has fewer than 24 days, the verdict is an early lead. That number comes from guidance on single person trials, which need a few dozen events before an effect is worth trusting.

Every number the network touches is wrapped in a small object: its value, a slot for its gradient, the inputs it came from, and a closure that knows how to push a gradient back to those inputs. Doing arithmetic on these objects builds a graph as a side effect. By the time the loss is computed, the graph records every multiply, add and squash that produced it.

The engine is scalar, like micrograd. One training step over 96 days with a 13 · 8 · 1 network builds 14,062 nodes, then throws them away. Parameters are the only nodes that live across steps. Each day's inputs are nodes as well, which is what makes the credits possible.

If the loss L depends on a node u, and u feeds into v, then the chain rule says dL/du picks up dL/dv times dv/du. A node used in several places sums the contributions from each use. That sum is why every backward closure does +=, never =.

dL/du = Σ over children v of  (dL/dv) · (dv/du)

Reverse mode visits nodes so that every node is finished before its inputs. The engine builds a topological order with an iterative depth first search (no recursion, so a long graph cannot blow the stack), sets dL/dL = 1 on the loss, and calls each closure from the loss back to the leaves. One backward sweep yields the gradient for every parameter and every input at about the cost of one extra forward pass. Forward mode would need one pass per parameter.

An op only needs to know its own derivative. The chain rule does the rest. These are the five ops the whole tool runs on, with out as the result and g as the gradient arriving at out.

mul(a, b)     a.grad += b · g        b.grad += a · g
scale(a, k)   a.grad += k · g                       k is a plain number
sum(xs)       x.grad += g  for every x
tanh(a)       a.grad += (1 − out²) · g
softplus(a)   a.grad += sigmoid(a) · g              softplus(a) = log(1 + eᵃ)

An n ary sum instead of chained binary adds keeps the graph shallow: a neuron with 8 inputs is one sum node, not a chain of 8. Softplus is computed as max(a, 0) + log1p(e^−|a|) so large logits never overflow, and its local gradient uses a sigmoid written to stay stable on both sides of zero.

The MLP maps one day's inputs to a logit z, and the risk is sigmoid(z). The hidden layer is h = tanh(W · x + b) with weights drawn with variance 1 / fan in, and the output layer is linear. Drop the hidden layer and the same code is logistic regression, which is the single factor baseline.

loss  = softplus(z) if the day was fine, softplus(−z) if it was bad
total = mean over days + λ · Σ w²        λ = 0.03, weights only
adam  m ← 0.9 m + 0.1 g    s ← 0.999 s + 0.001 g²    p ← p − 0.03 · m̂ / (√ŝ + 1e−8)
sgd   v ← 0.9 v − 0.1 g    p ← p + v

Training is full batch, 150 epochs for the logistic model and 250 for the MLP, in slices of about 10 ms per animation frame so the page stays responsive. The penalty is strong on purpose. With about a hundred training days and a dozen inputs, a lighter penalty let the MLP memorise the training days and lose to the base rate on held out ones; 0.03 was the lightest setting that stopped that on the sample and its reseeded copies.

Training only needs gradients for the weights, but the inputs are graph nodes too, so one backward pass from a day's logit also gives dz/dx for every input. The chain rule through the sigmoid turns that into the change in risk:

∂p/∂xⱼ = p (1 − p) · ∂z/∂xⱼ
credit j = mean over days of |∂p/∂xⱼ|     direction = sign of the mean slope

Each day is scored by the fold model that never trained on it. Since inputs are z scores, a credit reads as the change in risk for a one standard deviation swing, shown in percentage points. The page also counts how many of the five fold models agree on the direction.

The top two factors that are numbers or yes or no get a partial dependence map: set both to each point on a 24 by 24 grid for every day, average the five models' risk, and paint it. The joint share is Friedman's H²: the part of the map left after subtracting each factor's own average curve.

The button compares autograd against a central difference from a separate numeric loss that never builds a graph, for every weight and for every input of every training day:

numeric = (L(v + h) − L(v − h)) / 2h                h = 1e−5
error   = |auto − numeric| / max(|auto|, |numeric|, 1e−6)

Below 1e−4 passes. The floor is 1e−6 rather than smaller because the check runs on a trained model, where the penalised loss sits near a minimum and many weight gradients are close to zero. There a tiny absolute difference turns into a large relative one without anything being wrong.

Measured by running this exact code in Node at the defaults: 8 hidden units, Adam, L2 0.03, five folds of consecutive days. The sample is 120 synthetic days. A day is bad with probability 0.85 when sleep is under 6.3 hours and a meal was skipped, and 0.06 otherwise. Stress is made to rise after short nights, so it is correlated with the cause without being one. Shuffled is the same file with the bad day column randomly permuted, which destroys any real link.

On every seed the two planted causes take the top two credits, and stress never does. Seed 8 still gets no signal: two of its five held out stretches were scored worse than the base rate, so the gain does not clear its own spread. That is the gate doing its job on a real but noisy effect with 29 bad days, and a reminder that a clean verdict needs more data than a hunch does.

On the sample the corner with short sleep and a skipped meal reaches 72% risk, against 39% for short sleep alone and 3 to 7% after long sleep. The MLP beats the logistic model by only 0.008 log-loss. That is honest rather than disappointing: an AND rule is close to additive once the sigmoid squashes it, so a single layer already gets most of it, and the hidden layer has little left to add.

A 13 · 8 · 1 epoch takes about 1.3 ms in Node, so a full screen of 2,000 epochs is under 3 seconds of compute, spread over frames. Gradient check after training: worst error 2.1e−5 over 121 weights and 2.2e−6 over 1,248 input slopes.

• Association, not cause. A factor that rides along with the real cause can take credit for it. The only test is changing one thing for a few weeks and logging what happens. Not medical advice.
• Small samples are the norm. With fewer than about 24 bad days the tool says so, and even above that, five folds give a rough spread, not a confidence interval.
• Many inputs, few days. Each category level, blank flag and yesterday column is another input. Turning on yesterday's values doubles them, and on the sample it pushes the spread past the gain.
• Only one hidden layer of up to 12 units, and only binary outcomes. A numeric target is turned into bad or fine with a threshold you choose.
• Long format exports, with one row per factor per day, are not pivoted. The tool expects one row per day, or rows it can merge by date.
• Z scores and blank flags are computed on all days rather than per fold. They never see the labels, so the leak is small, but it is not zero.