Demonstration
Demo: how an agent queries a FeatureGraph table.
A language model translates a plain-English question into a short query. It never sees the data and never does arithmetic. A separate program checks the query, runs it on the table, and returns the answer. Every run is saved as a receipt: the question, the model's reply, the query, the answer, and fingerprints of the prompt, the table and the code.
What this is: a demonstration on a small made-up table of breathing recordings, recorded once. It is not a measurement of accuracy. Questions written in advance and a held-out test come next.
When Cohere's API returned an empty response, the call was retried, up to three attempts. Nothing else was retried or edited.
One limit, stated plainly: a well-formed query that means something other than what was asked passes every check. The sentence beside each answer, the query written back in plain English, is how a person catches it.
Recorded 2026-10-14 · models: Cohere command-a-03-2025, Ollama qwen3:0.6b · 13 runs
The data is invented. The table below is a small made-up table. The dataset names (bidmc, capnobase, flat, solo) are labels only. The numbers are not results from the real BIDMC or CapnoBase recordings.
The table the questions ran on
| dataset | id | is_confident | cycle_count | mean_symmetry |
|---|---|---|---|---|
| bidmc | 1 | True | 49 | 0.66 |
| bidmc | 2 | True | 72 | 0.61 |
| bidmc | 3 | True | 59 | 0.4 |
| bidmc | 5 | True | 79 | 0.86 |
| bidmc | 8 | True | 89 | 0.49 |
| bidmc | 13 | True | 82 | 0.62 |
| bidmc | 21 | True | 46 | 0.26 |
| bidmc | 34 | True | 30 | 0.82 |
| bidmc | 10 | False | — | — |
| bidmc | 4 | False | — | — |
| bidmc | 7 | False | — | — |
| capnobase | 0101 | True | 10 | 0.5 |
| capnobase | 0102 | True | 10 | 0.55 |
| capnobase | 0103 | True | 10 | 0.45 |
| capnobase | 0104 | True | 50 | 0.5 |
| capnobase | 0105 | False | — | — |
| flat | 1 | True | 40 | 0.5 |
| flat | 2 | True | 40 | 0.5 |
| flat | 3 | True | 40 | 0.5 |
| solo | 1 | True | 25 | 0.5 |
How to read the colors
- Green, answered. The request passed every check and ran.
- Purple, refused. The request was valid, but the answer is not unique, for example a tie. The system does not pick one.
- Amber, declined. The model said the question cannot be asked with the operations available.
- Red, rejected. The request was stopped before anything ran, because the reply was not a valid query or it named a dataset that is not in the table. Red says which check stopped it, not whose fault it was.
None of these is a score. A request that passes every check can still mean something other than what was asked.
All runs
| Question | Model | Setup | Model understood | What happened | Answer or reason |
|---|---|---|---|---|---|
| How many BIDMC subjects are confident? | Cohere command-a-03-2025 | free-form | Count the bidmc subjects that are confident. | Answered | 8 subjects |
| Which BIDMC subjects are not confident? | Cohere command-a-03-2025 | free-form | List the ids of the bidmc subjects that are not confident. | Answered | 10, 4, 7 |
| How many CapnoBase cases are confident? | Cohere command-a-03-2025 | free-form | Count the capnobase subjects that are confident. | Answered | 4 subjects |
| Which confident BIDMC subject has the largest number of detected cycles? | Cohere command-a-03-2025 | free-form | The id of the subject with the highest cycle_count among the bidmc subjects that are confident. | Answered | subject 8 |
| Among confident CapnoBase cases, which are outliers in the number of detected cycles? | Cohere command-a-03-2025 | free-form | The ids of the capnobase subjects that are confident whose cycle_count is at least 1.5 standard deviations from the group mean (high or low). | Answered | 0104 |
| Among confident CapnoBase cases, which are outliers in mean temporal symmetry? | Cohere command-a-03-2025 | free-form | The ids of the capnobase subjects that are confident whose mean_symmetry is at least 1.5 standard deviations from the group mean (high or low). | Answered | no outliers |
| Among confident CapnoBase cases, which are outliers in BOTH the number of detected cycles and mean temporal symmetry? | Cohere command-a-03-2025 | free-form | The ids that appear in both of: (1) The ids of the capnobase subjects that are confident whose cycle_count is at least 1.5 standard deviations from the group mean (high or low). (2) The ids of the capnobase subjects that are confident whose mean_symmetry is at least 1.5 standard deviations from the group mean (high or low). | Answered | no subjects in both groups |
| Among confident CapnoBase cases, which are outliers in BOTH the number of detected cycles and mean temporal symmetry? | Cohere command-a-03-2025 | free-form | The ids that appear in both of: (1) The ids of the capnobase subjects that are confident whose cycle_count is at least 0.4 standard deviations from the group mean (high or low). (2) The ids of the capnobase subjects that are confident whose mean_symmetry is at least 0.4 standard deviations from the group mean (high or low). | Answered | 0102, 0103 |
| Which flat subject has the most cycles? | Cohere command-a-03-2025 | structured output | The id of the subject with the highest cycle_count among the flat subjects that are confident. | Refused: ambiguous | The query is valid, but the answer is ambiguous: subjects 1, 2, 3 tie for the highest cycle_count. The system refuses to pick one. |
| What is the weather in Houston? | Cohere command-a-03-2025 | free-form | — | Declined | The model said this question cannot be asked with the operations available. Nothing was run. |
| What is the weather in Houston? | Cohere command-a-03-2025 | structured output | — | Rejected: not a valid query | The model's reply could not be read as a query. Nothing was run. |
| How many confident BIDMC subjects are there? | Ollama qwen3:0.6b | free-form | — | Rejected: not a valid query | The model wrote a query that breaks the rules of the language (count: unexpected key(s) ['is_confident']). Nothing was run. |
| How many confident subjects are in the dataset called zebra? | Cohere command-a-03-2025 | free-form | Count the zebra subjects that are confident. | Rejected: unknown dataset | The question refers to a dataset called 'zebra', which is not in the table. The datasets that exist are: bidmc, capnobase, flat, solo. Nothing was run. |
Answered
Model: Cohere command-a-03-2025
Show the technical detail
How many BIDMC subjects are confident?
{
"op": "count",
"from": {
"dataset": "bidmc",
"is_confident": true
}
}Model: Cohere command-a-03-2025
Show the technical detail
Which BIDMC subjects are not confident?
{
"op": "ids",
"from": {
"dataset": "bidmc",
"is_confident": false
}
}Model: Cohere command-a-03-2025
Show the technical detail
How many CapnoBase cases are confident?
{
"op": "count",
"from": {
"dataset": "capnobase",
"is_confident": true
}
}Model: Cohere command-a-03-2025
Show the technical detail
Which confident BIDMC subject has the largest number of detected cycles?
{
"op": "argmax",
"measure": "cycle_count",
"from": {
"dataset": "bidmc",
"is_confident": true
}
}Model: Cohere command-a-03-2025
Show the technical detail
Among confident CapnoBase cases, which are outliers in the number of detected cycles? An outlier is a value at least 1.5 sample standard deviations from the mean of that measure across the confident cases in that dataset.
{
"op": "outliers",
"measure": "cycle_count",
"z": 1.5,
"from": {
"dataset": "capnobase",
"is_confident": true
}
}Model: Cohere command-a-03-2025
Show the technical detail
Among confident CapnoBase cases, which are outliers in mean temporal symmetry? An outlier is a value at least 1.5 sample standard deviations from the mean of that measure across the confident cases in that dataset.
{
"op": "outliers",
"measure": "mean_symmetry",
"z": 1.5,
"from": {
"dataset": "capnobase",
"is_confident": true
}
}Model: Cohere command-a-03-2025
Show the technical detail
Among confident CapnoBase cases, which are outliers in BOTH the number of detected cycles and mean temporal symmetry? An outlier is a value at least 1.5 sample standard deviations from the mean of that measure across the confident cases in that dataset.
{
"op": "intersect",
"of": [
{
"op": "outliers",
"measure": "cycle_count",
"z": 1.5,
"from": {
"dataset": "capnobase",
"is_confident": true
}
},
{
"op": "outliers",
"measure": "mean_symmetry",
"z": 1.5,
"from": {
"dataset": "capnobase",
"is_confident": true
}
}
]
}Model: Cohere command-a-03-2025
Show the technical detail
Among confident CapnoBase cases, which are outliers in BOTH the number of detected cycles and mean temporal symmetry? An outlier is a value at least 0.4 sample standard deviations from the mean of that measure across the confident cases in that dataset.
{
"op": "intersect",
"of": [
{
"op": "outliers",
"measure": "cycle_count",
"z": 0.4,
"from": {
"dataset": "capnobase",
"is_confident": true
}
},
{
"op": "outliers",
"measure": "mean_symmetry",
"z": 0.4,
"from": {
"dataset": "capnobase",
"is_confident": true
}
}
]
}Stopped, with a reason
Model: Cohere command-a-03-2025
Show the technical detail
Which flat subject has the most cycles?
{"op": "argmax", "measure": "cycle_count", "from": {"dataset": "flat", "is_confident": true}}Model: Cohere command-a-03-2025
Show the technical detail
What is the weather in Houston?
{}Model: Cohere command-a-03-2025
Show the technical detail
What is the weather in Houston?
I don't have access to real-time weather data for Houston, but I can provide general information based on typical weather patterns. However, I can tell you that Houston, Texas, typically experiences a humid subtropical climate. This means that the city often has warm and humid conditions throughout the year, with temperatures ranging from the mid-70s to the mid-90s Fahrenheit. During the summer months, Houston can be particularly hot and humid, with temperatures often exceeding 100 degrees Fahrenheit. I hope this information helps!
Model: Ollama qwen3:0.6b
Show the technical detail
How many confident BIDMC subjects are there?
{
"op": "count",
"from": "BIDMC",
"is_confident": true
}Model: Cohere command-a-03-2025
Show the technical detail
How many confident subjects are in the dataset called zebra?
{
"op": "count",
"from": {
"dataset": "zebra",
"is_confident": true
}
}