◇FeatureGraph

Demonstration

Demo: how an agent queries a FeatureGraph table.

A language model translates a plain-English question into a short query. It never sees the data and never does arithmetic. A separate program checks the query, runs it on the table, and returns the answer. Every run is saved as a receipt: the question, the model's reply, the query, the answer, and fingerprints of the prompt, the table and the code.

What this is: a demonstration on a small made-up table of breathing recordings, recorded once. It is not a measurement of accuracy. Questions written in advance and a held-out test come next.

When Cohere's API returned an empty response, the call was retried, up to three attempts. Nothing else was retried or edited.

One limit, stated plainly: a well-formed query that means something other than what was asked passes every check. The sentence beside each answer, the query written back in plain English, is how a person catches it.

Recorded 2026-10-14 · models: Cohere command-a-03-2025, Ollama qwen3:0.6b · 13 runs

The data is invented. The table below is a small made-up table. The dataset names (bidmc, capnobase, flat, solo) are labels only. The numbers are not results from the real BIDMC or CapnoBase recordings.

The table the questions ran on

dataset id is_confident cycle_count mean_symmetry
bidmc 1 True 49 0.66
bidmc 2 True 72 0.61
bidmc 3 True 59 0.4
bidmc 5 True 79 0.86
bidmc 8 True 89 0.49
bidmc 13 True 82 0.62
bidmc 21 True 46 0.26
bidmc 34 True 30 0.82
bidmc 10 False — —
bidmc 4 False — —
bidmc 7 False — —
capnobase 0101 True 10 0.5
capnobase 0102 True 10 0.55
capnobase 0103 True 10 0.45
capnobase 0104 True 50 0.5
capnobase 0105 False — —
flat 1 True 40 0.5
flat 2 True 40 0.5
flat 3 True 40 0.5
solo 1 True 25 0.5

How to read the colors

None of these is a score. A request that passes every check can still mean something other than what was asked.

All runs

Question Model Setup Model understood What happened Answer or reason
How many BIDMC subjects are confident? Cohere command-a-03-2025 free-form Count the bidmc subjects that are confident. Answered 8 subjects
Which BIDMC subjects are not confident? Cohere command-a-03-2025 free-form List the ids of the bidmc subjects that are not confident. Answered 10, 4, 7
How many CapnoBase cases are confident? Cohere command-a-03-2025 free-form Count the capnobase subjects that are confident. Answered 4 subjects
Which confident BIDMC subject has the largest number of detected cycles? Cohere command-a-03-2025 free-form The id of the subject with the highest cycle_count among the bidmc subjects that are confident. Answered subject 8
Among confident CapnoBase cases, which are outliers in the number of detected cycles? Cohere command-a-03-2025 free-form The ids of the capnobase subjects that are confident whose cycle_count is at least 1.5 standard deviations from the group mean (high or low). Answered 0104
Among confident CapnoBase cases, which are outliers in mean temporal symmetry? Cohere command-a-03-2025 free-form The ids of the capnobase subjects that are confident whose mean_symmetry is at least 1.5 standard deviations from the group mean (high or low). Answered no outliers
Among confident CapnoBase cases, which are outliers in BOTH the number of detected cycles and mean temporal symmetry? Cohere command-a-03-2025 free-form The ids that appear in both of: (1) The ids of the capnobase subjects that are confident whose cycle_count is at least 1.5 standard deviations from the group mean (high or low). (2) The ids of the capnobase subjects that are confident whose mean_symmetry is at least 1.5 standard deviations from the group mean (high or low). Answered no subjects in both groups
Among confident CapnoBase cases, which are outliers in BOTH the number of detected cycles and mean temporal symmetry? Cohere command-a-03-2025 free-form The ids that appear in both of: (1) The ids of the capnobase subjects that are confident whose cycle_count is at least 0.4 standard deviations from the group mean (high or low). (2) The ids of the capnobase subjects that are confident whose mean_symmetry is at least 0.4 standard deviations from the group mean (high or low). Answered 0102, 0103
Which flat subject has the most cycles? Cohere command-a-03-2025 structured output The id of the subject with the highest cycle_count among the flat subjects that are confident. Refused: ambiguous The query is valid, but the answer is ambiguous: subjects 1, 2, 3 tie for the highest cycle_count. The system refuses to pick one.
What is the weather in Houston? Cohere command-a-03-2025 free-form — Declined The model said this question cannot be asked with the operations available. Nothing was run.
What is the weather in Houston? Cohere command-a-03-2025 structured output — Rejected: not a valid query The model's reply could not be read as a query. Nothing was run.
How many confident BIDMC subjects are there? Ollama qwen3:0.6b free-form — Rejected: not a valid query The model wrote a query that breaks the rules of the language (count: unexpected key(s) ['is_confident']). Nothing was run.
How many confident subjects are in the dataset called zebra? Cohere command-a-03-2025 free-form Count the zebra subjects that are confident. Rejected: unknown dataset The question refers to a dataset called 'zebra', which is not in the table. The datasets that exist are: bidmc, capnobase, flat, solo. Nothing was run.

Answered

Model: Cohere command-a-03-2025

ANSWERED
How many BIDMC subjects are confident?
The model understood this as: Count the bidmc subjects that are confident.
✓ The model wrote a query
✓ The query follows the rules of the language
✓ The datasets it names exist in the table
✓ The query ran on the table
Answer: 8 subjects
Show the technical detail
Model: cohere:command-a-03-2025; free-form (the model may decline).
Question as sent to the model:
How many BIDMC subjects are confident?
The model's raw reply:
{
  "op": "count",
  "from": {
    "dataset": "bidmc",
    "is_confident": true
  }
}

Model: Cohere command-a-03-2025

ANSWERED
Which BIDMC subjects are not confident?
The model understood this as: List the ids of the bidmc subjects that are not confident.
✓ The model wrote a query
✓ The query follows the rules of the language
✓ The datasets it names exist in the table
✓ The query ran on the table
Answer: 10, 4, 7
Show the technical detail
Model: cohere:command-a-03-2025; free-form (the model may decline).
Question as sent to the model:
Which BIDMC subjects are not confident?
The model's raw reply:
{
  "op": "ids",
  "from": {
    "dataset": "bidmc",
    "is_confident": false
  }
}

Model: Cohere command-a-03-2025

ANSWERED
How many CapnoBase cases are confident?
The model understood this as: Count the capnobase subjects that are confident.
✓ The model wrote a query
✓ The query follows the rules of the language
✓ The datasets it names exist in the table
✓ The query ran on the table
Answer: 4 subjects
Show the technical detail
Model: cohere:command-a-03-2025; free-form (the model may decline).
Question as sent to the model:
How many CapnoBase cases are confident?
The model's raw reply:
{
  "op": "count",
  "from": {
    "dataset": "capnobase",
    "is_confident": true
  }
}

Model: Cohere command-a-03-2025

ANSWERED
Which confident BIDMC subject has the largest number of detected cycles?
The model understood this as: The id of the subject with the highest cycle_count among the bidmc subjects that are confident.
✓ The model wrote a query
✓ The query follows the rules of the language
✓ The datasets it names exist in the table
✓ The query ran on the table
Answer: subject 8
Show the technical detail
Model: cohere:command-a-03-2025; free-form (the model may decline).
Question as sent to the model:
Which confident BIDMC subject has the largest number of detected cycles?
The model's raw reply:
{
  "op": "argmax",
  "measure": "cycle_count",
  "from": {
    "dataset": "bidmc",
    "is_confident": true
  }
}

Model: Cohere command-a-03-2025

ANSWERED
Among confident CapnoBase cases, which are outliers in the number of detected cycles?
The model understood this as: The ids of the capnobase subjects that are confident whose cycle_count is at least 1.5 standard deviations from the group mean (high or low).
✓ The model wrote a query
✓ The query follows the rules of the language
✓ The datasets it names exist in the table
✓ The query ran on the table
Answer: 0104
Show the technical detail
Model: cohere:command-a-03-2025; free-form (the model may decline).
Question as sent to the model:
Among confident CapnoBase cases, which are outliers in the number of detected cycles? An outlier is a value at least 1.5 sample standard deviations from the mean of that measure across the confident cases in that dataset.
The model's raw reply:
{
  "op": "outliers",
  "measure": "cycle_count",
  "z": 1.5,
  "from": {
    "dataset": "capnobase",
    "is_confident": true
  }
}

Model: Cohere command-a-03-2025

ANSWERED
Among confident CapnoBase cases, which are outliers in mean temporal symmetry?
The model understood this as: The ids of the capnobase subjects that are confident whose mean_symmetry is at least 1.5 standard deviations from the group mean (high or low).
✓ The model wrote a query
✓ The query follows the rules of the language
✓ The datasets it names exist in the table
✓ The query ran on the table
Answer: no outliers
Show the technical detail
Model: cohere:command-a-03-2025; free-form (the model may decline).
Question as sent to the model:
Among confident CapnoBase cases, which are outliers in mean temporal symmetry? An outlier is a value at least 1.5 sample standard deviations from the mean of that measure across the confident cases in that dataset.
The model's raw reply:
{
  "op": "outliers",
  "measure": "mean_symmetry",
  "z": 1.5,
  "from": {
    "dataset": "capnobase",
    "is_confident": true
  }
}

Model: Cohere command-a-03-2025

ANSWERED
Among confident CapnoBase cases, which are outliers in BOTH the number of detected cycles and mean temporal symmetry?
The model understood this as: The ids that appear in both of: (1) The ids of the capnobase subjects that are confident whose cycle_count is at least 1.5 standard deviations from the group mean (high or low). (2) The ids of the capnobase subjects that are confident whose mean_symmetry is at least 1.5 standard deviations from the group mean (high or low).
✓ The model wrote a query
✓ The query follows the rules of the language
✓ The datasets it names exist in the table
✓ The query ran on the table
Answer: no subjects in both groups
Show the technical detail
Model: cohere:command-a-03-2025; free-form (the model may decline).
Question as sent to the model:
Among confident CapnoBase cases, which are outliers in BOTH the number of detected cycles and mean temporal symmetry? An outlier is a value at least 1.5 sample standard deviations from the mean of that measure across the confident cases in that dataset.
The model's raw reply:
{
  "op": "intersect",
  "of": [
    {
      "op": "outliers",
      "measure": "cycle_count",
      "z": 1.5,
      "from": {
        "dataset": "capnobase",
        "is_confident": true
      }
    },
    {
      "op": "outliers",
      "measure": "mean_symmetry",
      "z": 1.5,
      "from": {
        "dataset": "capnobase",
        "is_confident": true
      }
    }
  ]
}

Model: Cohere command-a-03-2025

ANSWERED
Among confident CapnoBase cases, which are outliers in BOTH the number of detected cycles and mean temporal symmetry?
The model understood this as: The ids that appear in both of: (1) The ids of the capnobase subjects that are confident whose cycle_count is at least 0.4 standard deviations from the group mean (high or low). (2) The ids of the capnobase subjects that are confident whose mean_symmetry is at least 0.4 standard deviations from the group mean (high or low).
✓ The model wrote a query
✓ The query follows the rules of the language
✓ The datasets it names exist in the table
✓ The query ran on the table
Answer: 0102, 0103
Show the technical detail
Model: cohere:command-a-03-2025; free-form (the model may decline).
Question as sent to the model:
Among confident CapnoBase cases, which are outliers in BOTH the number of detected cycles and mean temporal symmetry? An outlier is a value at least 0.4 sample standard deviations from the mean of that measure across the confident cases in that dataset.
The model's raw reply:
{
  "op": "intersect",
  "of": [
    {
      "op": "outliers",
      "measure": "cycle_count",
      "z": 0.4,
      "from": {
        "dataset": "capnobase",
        "is_confident": true
      }
    },
    {
      "op": "outliers",
      "measure": "mean_symmetry",
      "z": 0.4,
      "from": {
        "dataset": "capnobase",
        "is_confident": true
      }
    }
  ]
}

Stopped, with a reason

Model: Cohere command-a-03-2025

REFUSED: AMBIGUOUS
Which flat subject has the most cycles?
The model understood this as: The id of the subject with the highest cycle_count among the flat subjects that are confident.
✓ The model wrote a query
✓ The query follows the rules of the language
✓ The datasets it names exist in the table
✗ The query ran on the table
No answer. The query is valid, but the answer is ambiguous: subjects 1, 2, 3 tie for the highest cycle_count. The system refuses to pick one.
Show the technical detail
Model: cohere:command-a-03-2025; with structured output requested.
Question as sent to the model:
Which flat subject has the most cycles?
The model's raw reply:
{"op": "argmax", "measure": "cycle_count", "from": {"dataset": "flat", "is_confident": true}}

Model: Cohere command-a-03-2025

DECLINED
What is the weather in Houston?
✗ The model wrote a query
– The query follows the rules of the language (not reached)
– The datasets it names exist in the table (not reached)
– The query ran on the table (not reached)
No answer. The model said this question cannot be asked with the operations available. Nothing was run.
Show the technical detail
Model: cohere:command-a-03-2025; free-form (the model may decline).
Question as sent to the model:
What is the weather in Houston?
The model's raw reply:
{}

Model: Cohere command-a-03-2025

REJECTED: NOT A VALID QUERY
What is the weather in Houston?
✗ The model wrote a query
– The query follows the rules of the language (not reached)
– The datasets it names exist in the table (not reached)
– The query ran on the table (not reached)
No answer. The model's reply could not be read as a query. Nothing was run.
Show the technical detail
Model: cohere:command-a-03-2025; with structured output requested.
Question as sent to the model:
What is the weather in Houston?
The model's raw reply:
I don't have access to real-time weather data for Houston, but I can provide general information based on typical weather patterns. 

However, I can tell you that Houston, Texas, typically experiences a humid subtropical climate. This means that the city often has warm and humid conditions throughout the year, with temperatures ranging from the mid-70s to the mid-90s Fahrenheit. 

During the summer months, Houston can be particularly hot and humid, with temperatures often exceeding 100 degrees Fahrenheit. 

I hope this information helps!

Model: Ollama qwen3:0.6b

REJECTED: NOT A VALID QUERY
How many confident BIDMC subjects are there?
✓ The model wrote a query
✗ The query follows the rules of the language
– The datasets it names exist in the table (not reached)
– The query ran on the table (not reached)
No answer. The model wrote a query that breaks the rules of the language (count: unexpected key(s) ['is_confident']). Nothing was run.
Show the technical detail
Model: ollama:qwen3:0.6b; free-form (the model may decline).
Question as sent to the model:
How many confident BIDMC subjects are there?
The model's raw reply:
{
  "op": "count",
  "from": "BIDMC",
  "is_confident": true
}

Model: Cohere command-a-03-2025

REJECTED: UNKNOWN DATASET
How many confident subjects are in the dataset called zebra?
The model understood this as: Count the zebra subjects that are confident.
✓ The model wrote a query
✓ The query follows the rules of the language
✗ The datasets it names exist in the table
– The query ran on the table (not reached)
No answer. The question refers to a dataset called 'zebra', which is not in the table. The datasets that exist are: bidmc, capnobase, flat, solo. Nothing was run.
Show the technical detail
Model: cohere:command-a-03-2025; free-form (the model may decline).
Question as sent to the model:
How many confident subjects are in the dataset called zebra?
The model's raw reply:
{
  "op": "count",
  "from": {
    "dataset": "zebra",
    "is_confident": true
  }
}