A Little Time Doesn’t Hurt

What happens when you tell an LLM the time of day? Does it fixate on the time and perform worse at its job?

To find out, I forked τ²-bench (Barres et al., 2025), an open-source benchmark from Sierra. I ran an experiment to observe how telling the model the time of day affected its performance on this benchmark.

Motivation#

Over this past year, I’ve wrapped up my day by writing down my reflections and sending them to Claude. Occasionally, I continued those conversations the next morning (eg: new realizations after sleeping on something). Without fail, Claude would respond and tell me to close my laptop and go to sleep. It had no idea what time it was. I call this temporal disorientation.

Here’s what it looks like:

This behavior annoys me. At best, it’s a distraction. At worst, it makes me wonder if Claude isn’t working as hard because it thinks I’m winding down for the day. I’m not alone in feeling this way; a quick Google search surfaces many instances of people complaining about this exact behavior.

A simple solution I thought of was to include a short timestamp at each conversation turn. I asked Claude and other models about this, and they weren’t too enthusiastic. They warned of meaningfully degraded model performance because the model’s attention would be polluted with timestamps. Claude even claimed it would cause models to fixate on the time of day and talk about that instead of the topic at hand.

Those explanations didn’t satisfy me, so I decided to find out what would happen if I included timestamps in the conversation.

Setup: τ²-bench#

I wanted to reuse an existing, established eval rather than invent my own for this experiment. τ²-bench caught my attention because it grades multi-turn conversations, its rewards are deterministic, and it doesn’t cost a small fortune to run. That first attribute was key. While many benchmarks evaluate coding ability or textbook knowledge, I needed a benchmark that simulated user conversations so I could see what would happen once timestamps entered the chat.

Test conditions#

My experiment included a control and three test conditions:

1. Control: τ²-bench without modification
2. Test 1: ’time-only’, prepend each user message with [HH:mm:ss]. Example: [18:58:31] I was chatting with a friend about...
3. Test 2: ’notice-only’, add an instruction to the system prompt that looks like this: The system injects the time-of-day at the start of each user message... (full text)
4. Test 3: combines tests 1 and 2; user messages include the time-of-day and the system prompt tells the model to expect them.

Why test time-of-day ([HH:mm:ss]) instead of a full timestamp? My intuition is that time-of-day is all a model needs to cure temporal disorientation. I don’t think year, month, or day would be impactful if what I want to solve is Claude telling me to ‘get some rest’ when it’s almost lunch time.

Time simulation#

To control the time-of-day stamp across test and control, I simulate time-of-day rather than read the system clock. For a set of matched simulations, I use a random seed to pick the initial time-of-day. The simulated amount of time in between each message is also randomly picked at each turn using that same seed. So, for the two test conditions in a set that inject the time-of-day, the time-of-day is exactly the same at each conversation turn within that pair.

Domain#

τ²-bench features various ‘domains’; each domain is a vertical of customer support. I selected the telecom domain for my experiment because my evaluations showed that it presented a challenge for the models I was evaluating. This domain consists of 114 tasks. I ran the full telecom domain 8 times (k=8) using a different seed per run, with the same 8 seeds reused across conditions and models. This setup means each condition had 114 tasks × 8 seeds = 912 simulations per model.

Models#

Speaking of models, I selected GPT-5.6 Luna, Muse Spark 1.2, and Gemma 4 12B (BF16 and W4A16). I wanted to use current models and models that wouldn’t cost a fortune to test. I also selected GPT-5.6 Luna to be the user simulator.

Results#

The experiment uses τ²-bench’s standard binary reward function to score each simulation. If the assistant successfully resolves the telecom support ticket, it gets a 1, otherwise 0.

Performance#

Here’s a quick look at each model’s average reward per condition.

Meta’s Muse Spark 1.2 scores highest by a wide margin. It’s also the newest and most expensive model in this analysis.

Test effects#

Here’s how the models performed on each test condition and the estimated impact each test condition had on model performance. The confidence intervals are computed using a two-way cluster bootstrap with 100,000 resamples of tasks and seeds.

The bolded rows highlight tests where the 95% CI excludes zero.

The data shows little statistically significant impact on model performance across all of the test conditions. Additionally, the models never commented on what time it was in their messages to the user.

Spend#

τ²-bench helpfully tracks token and dollar spend out of the box.

Muse Spark 1.2 pricing reflects standard pricing. The experiment was run on the cheaper Contributor tier.

What I find most interesting about this table is GPT-5.6 Luna’s constant efficiency. Spend more (for higher effort) and get more solved tickets for the same price per solved ticket.

Takeaways#

Claude’s predictions were shown to be false in this experiment. Model performance did not meaningfully degrade. The models did not fixate on what time it was.

Introducing the time-of-day stamp, a notice about the stamp, or both does not degrade model performance on τ²-bench’s telecom domain. This sounds boring on the surface, but it’s a promising result. This indicates that time-of-day stamps could be introduced without impact on model performance. I haven’t tested whether including these time-of-day stamps would cure temporal disorientation, but I have a hunch they would help.

Other reflections#

Gemma 4 12B exceeded my (low) expectations. Of all models, I was most worried about it being negatively impacted by the addition of the time-of-day stamp. While the quantized W4A16 checkpoint was the only model with consistently negative Δ values, its 95% CI still straddled 0. I was also impressed that W4A16 only suffered a modest performance penalty compared with the BF16 checkpoint.

GPT-5.6 Luna punched above its weight. It was the second-strongest performing model, but the cheapest frontier model by an order of magnitude. Paired with its relatively fast speed, I think it presents an appealing value proposition.

Methods#

My repo forked the upstream tau2-bench at 668d3bcd. My post-fork changes include:

• Adding support for the OpenAI Responses API
• Adding a new agent so I could inject time-of-day and the notice
• Adding a web viewer so I could quickly inspect simulations
• Adding chat to the web viewer (powered by Gemma 4 E4B running locally) so I could quickly locate certain simulations and scenarios

The Gemma 4 12B runs were executed on A40s hosted by Runpod. The BF16 and QAT W4A16 were the official checkpoints from Google.

Muse Spark 1.2 was run on the Meta Model API’s Contributor tier as it offered a significant discount (90%+).

Notice text#

The full notice injected into the system prompt:

The system injects the time-of-day at the start of each user message. It is formatted `HH:mm:ss`, local to the user's timezone, and denotes when the user sent their message.
Example: "[09:31:56] Hi, I need help with..."

Clock parameters#

Simulated clock parameters:

CLOCK_START_WINDOW = ("04:30:00", "22:30:00")
CLOCK_INTERVAL_SECONDS = (60, 900)

Time-mention detection#

The assistant messages were scanned with a case-insensitive, word-boundary time-of-day regex that checked for: morning, afternoon, evening, night, tonight, noon, midnight, midday, dawn, dusk, o’clock, \d{1,2}(:\d{2})?\s?[ap]m, “time of day”, “this hour”, early/late (morning|afternoon|evening|night).

Limits#

The main thing missing from this experiment is evidence demonstrating that temporal disorientation can be cured by including the time of day.

The seed diversity is low at k=8. While the number of simulations run per model per condition isn’t too low (n=912), the low number of seeds means that the models weren’t evaluated under a breadth of conditions. This was primarily driven by a motivation to keep costs low.

References#

• Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment. arXiv:2506.07982, 2025.

• Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv:2406.12045, 2024.