Claude vs ChatGPT vs Gemini for stock analysis: one 10-K, three models
Disclosure: I have no position in GOOGL. I also hold diversified funds (IWDA, QQQ, SPY, VWRL) that may include GOOGL. Positions checked on 3 November 2026.
I gave Claude, OpenAI’s GPT model and Gemini the same Alphabet 10-K and the same twelve questions, twice each. On pure number retrieval, Claude vs ChatGPT for stock analysis is a draw, and Gemini ties too. The six runs got 72 of 72 answers right against the filing, and every run declined the two questions the 10-K cannot answer. The models split on a definition. For instance, asked for free cash flow after stock-based compensation in a looser analyst prompt, the GPT run subtracted the $27,100 million total from the notes, while the other five runs used the $24,953 million line in the cash flow statement. Same filing, two answers, both printed in it.
The setup of the check
The input was Alphabet’s annual report on Form 10-K for fiscal 2025, filed on 5 February 2026, converted from the SEC’s HTML to a plain text file with each table row on one line. The code below rebuilds that file from EDGAR so that you can use the same input if you want to test it as well.
import hashlib
import os
import re
from html.parser import HTMLParser
import requests
# The SEC asks every script to identify itself. Set your own contact first:
# export SEC_USER_AGENT="Your Name you@example.com"
HEADERS = {"User-Agent": os.environ["SEC_USER_AGENT"]}
TEN_K = "https://www.sec.gov/Archives/edgar/data/1652044/000165204426000018/goog-20251231.htm"
def clean(s):
"""Collapse spaces, tabs, non-breaking and zero-width spaces."""
return re.sub(r"[ \t\xa0]+", " ", s).strip()
class FilingText(HTMLParser):
"""Plain text of a filing, one table row per line with cells joined by ' | '."""
def __init__(self):
super().__init__(convert_charrefs=True)
self.out, self.row, self.cell, self.skip = [], None, None, 0
def handle_starttag(self, tag, attrs):
if tag in ("script", "style", "ix:header"):
self.skip += 1
elif tag == "tr":
self.row = []
elif tag in ("td", "th") and self.row is not None:
self.cell = []
def handle_endtag(self, tag):
if tag in ("script", "style", "ix:header"):
self.skip -= 1
elif tag in ("td", "th") and self.cell is not None:
self.row.append(clean(" ".join(self.cell)))
self.cell = None
elif tag == "tr" and self.row is not None:
merged = []
for c in (c for c in self.row if c):
if merged and c in (")", "%", ")%"):
merged[-1] += c # glue closing brackets and % signs to their number
elif merged and merged[-1] in ("$", "(", "$("):
merged[-1] = c if merged[-1] == "$" else merged[-1] + c
else:
merged.append(c)
self.out.append("\n" + " | ".join(merged) + "\n")
self.row = None
elif tag in ("p", "div", "br", "li", "h1", "h2", "h3", "h4") and self.row is None:
self.out.append("\n")
def handle_data(self, data):
if self.skip:
return
if self.cell is not None:
self.cell.append(data)
elif self.row is None:
self.out.append(data)
def filing_text(url):
"""Download a filing from EDGAR and return its plain text."""
parser = FilingText()
parser.feed(requests.get(url, headers=HEADERS, timeout=60).text)
lines = []
for line in (clean(l) for l in "".join(parser.out).split("\n")):
if line or (lines and lines[-1]):
lines.append(line)
return "\n".join(lines)
text = filing_text(TEN_K)
with open("goog-10k-fy2025.txt", "w", encoding="utf-8") as f:
f.write(text)
print(f"{len(text):,} characters, {text.count(chr(10)) + 1:,} lines")
print("sha256", hashlib.sha256(text.encode()).hexdigest()[:16])
print([l for l in text.splitlines() if l.startswith("Net cash provided by operating")][-1])348,918 characters, 3,463 lines
sha256 fac05225ebdc645d
Net cash provided by operating activities | 101,746 | 125,299 | 164,713The models were ran through its command-line tool in non-interactive mode, from a folder holding only that file: Claude Code with claude-opus-5-5, OpenAI’s Codex CLI with gpt-6.1-sol and OpenCode 2.2.4 with gemini-4-argon.
I changed no reasoning or sampling settings, so each tool used its own defaults.
Test 1: twelve figures from the 10-K
The first prompt asks for ten figures and sets two traps. Alphabet’s 10-K says only that it expects to “significantly increase” spending on technical infrastructure in 2026, with no dollar amount, and it contains no figures for the quarter to 31 March 2026. A model that fills either gap from memory fails.
The file goog-10k-fy2025.txt in this folder is the text of Alphabet Inc.'s annual report on Form 10-K for the fiscal year ended December 31, 2025, as filed with the SEC on February 5, 2026. Tables are flattened to one row per line, with cells separated by "|".
Answer using only this file. Do not use the web or anything you remember about Alphabet. If the file does not contain the answer, write "NOT IN DOCUMENT" instead of a number.
Give amounts in millions of US dollars and share counts in millions, as plain numbers.
1. revenue_2025: total revenues for 2025.
2. operating_cash_flow_2025: net cash provided by operating activities, 2025.
3. capex_2025: purchases of property and equipment, 2025.
4. free_cash_flow_2025: net cash provided by operating activities minus purchases of property and equipment, 2025.
5. free_cash_flow_2024: the same calculation for 2024.
6. stock_based_compensation_2025: stock-based compensation expense as shown in the cash flow statement, 2025.
7. buybacks_paid_2025: cash paid for repurchases of stock, as shown in the cash flow statement, 2025.
8. dividends_paid_2025: dividend payments, as shown in the cash flow statement, 2025.
9. shares_outstanding_total: total shares of all classes outstanding at the latest date given on the cover page.
10. cash_and_marketable_securities_2025: cash, cash equivalents and marketable securities at December 31, 2025.
11. capex_guidance_2026: the dollar amount of capital expenditures Alphabet expects to spend in 2026.
12. free_cash_flow_q1_2026: free cash flow for the quarter ended March 31, 2026.
Reply with one JSON object and nothing else, with these twelve keys. Each value is an object {"value": <number or "NOT IN DOCUMENT">, "source": "<the statement or section, and the line you read>"}. For question 9 add a "date" field.Every run replied with the same twelve values. The check below holds those values, copied from each run’s saved JSON, and compares them with what Alphabet tagged in XBRL for the same 10-K. The share count is not in the SEC’s companyfacts file, so the code reads it from the cover-page sentence instead. The functions follow the SEC EDGAR API tutorial; the XBRL parser post explains how companyfacts is laid out.
import pandas as pd
NID = "NOT IN DOCUMENT"
QUESTIONS = ["revenue_2025", "operating_cash_flow_2025", "capex_2025", "free_cash_flow_2025",
"free_cash_flow_2024", "stock_based_compensation_2025", "buybacks_paid_2025",
"dividends_paid_2025", "shares_outstanding_total", "cash_and_marketable_securities_2025",
"capex_guidance_2026", "free_cash_flow_q1_2026"]
# Each run's "value" fields, copied from its saved JSON reply ($ millions, shares in millions)
ANSWERS = {
"Claude run 1": [402836, 164713, 91447, 73266, 72764, 24953, 45709, 10049, 12097, 126843, NID, NID],
"Claude run 2": [402836, 164713, 91447, 73266, 72764, 24953, 45709, 10049, 12097, 126843, NID, NID],
"GPT run 1": [402836, 164713, 91447, 73266, 72764, 24953, 45709, 10049, 12097, 126843, NID, NID],
"GPT run 2": [402836, 164713, 91447, 73266, 72764, 24953, 45709, 10049, 12097, 126843, NID, NID],
"Gemini run 1": [402836, 164713, 91447, 73266, 72764, 24953, 45709, 10049, 12097, 126843, NID, NID],
"Gemini run 2": [402836, 164713, 91447, 73266, 72764, 24953, 45709, 10049, 12097, 126843, NID, NID],
}
FACTS = requests.get("https://data.sec.gov/api/xbrl/companyfacts/CIK0001652044.json",
headers=HEADERS, timeout=30).json()["facts"]["us-gaap"]
ACCN = "0001652044-26-000018" # the fiscal 2025 10-K
def fact(concept, end, start=None):
"""Value in $ millions that the fiscal 2025 10-K tagged for `concept` and period."""
rows = pd.DataFrame(FACTS[concept]["units"]["USD"])
rows = rows[(rows["accn"] == ACCN) & (rows["end"] == end)]
if start:
rows = rows[rows["start"] == start]
return rows["val"].iloc[-1] / 1e6
def year(concept, y):
"""Full-year value for calendar year `y` (Alphabet's fiscal year)."""
return fact(concept, f"{y}-12-31", f"{y}-01-01")
ocf = {y: year("NetCashProvidedByUsedInOperatingActivities", y) for y in (2023, 2024, 2025)}
capex = {y: year("PaymentsToAcquirePropertyPlantAndEquipment", y) for y in (2023, 2024, 2025)}
cover = re.search(r"As of (\w+ \d+, \d{4}), there were ([\d,]+) million shares of Alphabet’s Class A stock "
r"outstanding, ([\d,]+) million shares of Alphabet’s Class B stock outstanding, and "
r"([\d,]+) million shares of the Alphabet’s Class C", text)
key = [year("Revenues", 2025), ocf[2025], capex[2025], ocf[2025] - capex[2025], ocf[2024] - capex[2024],
year("ShareBasedCompensation", 2025), year("PaymentsForRepurchaseOfCommonStock", 2025),
year("PaymentsOfDividends", 2025), sum(int(n.replace(",", "")) for n in cover.groups()[1:]),
fact("CashCashEquivalentsAndShortTermInvestments", "2025-12-31"),
NID if not re.search(r"In 2026, we expect[^.]*\$", text) else "dollar figure found",
NID if "March 31, 2026" not in text else "Q1 2026 found"]
def correct(said, truth):
"""An answer counts when it matches to the nearest $1 million, or both say NOT IN DOCUMENT."""
if isinstance(truth, str) or isinstance(said, str):
return said == truth
return abs(said - truth) < 1
score = pd.DataFrame({run: [correct(a, k) for a, k in zip(answers, key)] for run, answers in ANSWERS.items()},
index=QUESTIONS)
print(f"Cover-page share count: {cover.group(1)}")
print(score.sum().to_string())Cover-page share count: January 28, 2026
Claude run 1 12
Claude run 2 12
GPT run 1 12
GPT run 2 12
Gemini run 1 12
Gemini run 2 12Six runs, twelve out of twelve each, and nothing invented for 2026. Each run also named the line it read. Codex and OpenCode cited line numbers in the text file, Claude Code mostly quoted the table rows, and all three got the cover-page share count of 12,097 million by adding the three share classes themselves. Speed was the one clear difference. The two Claude runs took 20 and 25 seconds, the GPT runs 27 and 42, and the Gemini runs 87 and 111, according to the run logs. The second Gemini run’s log shows one search worth a look. Before answering the 2026 capex question, it searched the 10-K for “175|185”, two numbers that appear nowhere in the prompt. The search found only unrelated table cells, and the run answered “NOT IN DOCUMENT”. The model seems to have had a figure in mind from outside the file, checked for it and dropped it when the filing did not have it.
That result says as much about the setup as about the models. A clean text file with one table row per line, a question that names the statement to read, and a stated way to answer “not here” leave little room to go wrong. The second test drops the named statement.
Claude vs ChatGPT for stock analysis when the question is loose
Real prompts are vaguer than a list of line items, so the second test asked for a short analyst note on cash generation and left the definitions to the model:
The file goog-10k-fy2025.txt in this folder is the text of Alphabet Inc.'s annual report on Form 10-K for the fiscal year ended December 31, 2025, as filed with the SEC on February 5, 2026. Tables are flattened to one row per line, with cells separated by "|".
Answer using only this file. Do not use the web or anything you remember about Alphabet. If the file does not contain the answer, say so.
You are an equity analyst writing a short note on Alphabet's cash generation.
1. How much free cash flow did Alphabet generate in 2025, and how did it change from 2024?
2. How much cash did Alphabet return to shareholders in 2025, and what share of 2025 free cash flow was that?
3. What did stock-based compensation cost in 2025, and what was free cash flow after subtracting it?
4. Why did free cash flow grow so much less than operating cash flow in 2025?
5. How much does Alphabet plan to spend on capital expenditures in 2026?
Write the note in under 250 words. Then add one JSON object with these keys: fcf_2025, fcf_2024, shareholder_returns_2025, returns_pct_of_fcf_2025, sbc_2025, fcf_after_sbc_2025, capex_2026. Amounts in millions of US dollars, the percentage as a number (for example 61.2), and "NOT IN DOCUMENT" where the file has no answer.import matplotlib.pyplot as plt
# The analyst-note test: JSON values from each saved run, $ millions (percentage as a number)
NOTE_KEYS = ["fcf_2025", "fcf_2024", "shareholder_returns_2025", "returns_pct_of_fcf_2025", "sbc_2025",
"fcf_after_sbc_2025", "capex_2026"]
NOTES = {
"Claude run 1": [73266, 72764, 55758, 76.1, 24953, 48313, NID],
"Claude run 2": [73266, 72764, 55758, 76.1, 24953, 48313, NID],
"GPT run 1": [73266, 72764, 55758, 76.1, 27100, 46166, NID],
"GPT run 2": [73266, 72764, 55758, 76.1, 24953, 48313, NID],
"Gemini run 1": [73266, 72764, 55758, 76.1, 24953, 48313, NID],
"Gemini run 2": [73266, 72764, 55758, 76.1, 24953, 48313, NID],
}
fcf = {y: ocf[y] - capex[y] for y in ocf}
returns = year("PaymentsForRepurchaseOfCommonStock", 2025) + year("PaymentsOfDividends", 2025)
sbc_cash_flow = year("ShareBasedCompensation", 2025)
sbc_total = year("AllocatedShareBasedCompensationExpense", 2025) # Note 13, includes cash-settled awards
note_key = [fcf[2025], fcf[2024], returns, round(returns / fcf[2025] * 100, 1), sbc_cash_flow,
fcf[2025] - sbc_cash_flow, NID]
note_alt = [None, None, None, None, sbc_total, fcf[2025] - sbc_total, None]
def grade(said, truth, alt):
"""2 = matches the cash flow statement, 1 = matches another figure in the 10-K, 0 = neither."""
if correct(said, truth) or (isinstance(truth, float) and not isinstance(said, str)
and abs(said - truth) < 0.05):
return 2
return 1 if alt is not None and correct(said, alt) else 0
notes = pd.DataFrame({run: [grade(a, k, x) for a, k, x in zip(vals, note_key, note_alt)]
for run, vals in NOTES.items()}, index=NOTE_KEYS)
grid = pd.concat([score.astype(int) * 2, notes])
print(notes.to_string())
fig, ax = plt.subplots(figsize=(9, 8))
colors = plt.matplotlib.colors.ListedColormap(["#d62728", "#f2b134", "#2a9d8f"])
ax.imshow(grid.values, cmap=colors, vmin=0, vmax=2, aspect="auto")
ax.set_xticks(range(grid.shape[1]), grid.columns, rotation=30, ha="right")
ax.set_yticks(range(grid.shape[0]), grid.index)
ax.axhline(len(score) - 0.5, color="white", lw=4)
for i, j in zip(*((grid.values == 1).nonzero())):
ax.text(j, i, "Note 13\nfigure", ha="center", va="center", fontsize=8)
ax.set_title("Alphabet FY2025 10-K: 12 fixed questions (top) and the analyst note (bottom)\n"
"green = matches the cash flow statement or XBRL, amber = another figure in the 10-K")
plt.savefig("googl-10k-model-scorecard.png", dpi=150, bbox_inches="tight") Claude run 1 Claude run 2 GPT run 1 GPT run 2 Gemini run 1 Gemini run 2
fcf_2025 2 2 2 2 2 2
fcf_2024 2 2 2 2 2 2
shareholder_returns_2025 2 2 2 2 2 2
returns_pct_of_fcf_2025 2 2 2 2 2 2
sbc_2025 2 2 1 2 2 2
fcf_after_sbc_2025 2 2 1 2 2 2
capex_2026 2 2 2 2 2 2
The headline numbers agree everywhere. Free cash flow was $73.3 billion in 2025 against $72.8 billion in 2024, and buybacks plus dividends paid in cash came to $55.8 billion, or 76.1% of free cash flow. Every run said the 10-K gives no dollar figure for 2026 capex.
Stock-based compensation is where the definitions part, because the 10-K prints two figures for it. The cash flow statement adds back $24,953 million of stock-based compensation expense. Note 13 reports a total of $27,100 million, a figure that also covers awards Alphabet does not expect to settle in its own stock. GPT run 1 subtracted the larger figure and said so in its note, leaving $46,166 million. The other runs landed on $48,313 million. GPT run 2 used the cash flow line, so the same model on the same prompt chose differently from one run to the next. Both Gemini runs reported the two figures side by side, and the first one tied the gap to Waymo. The filing makes that link plausible without putting a number on it. Its accounting policies say stock-based compensation includes awards that may be settled in cash or in the stock of Other Bet companies. It also describes a $2.1 billion Waymo employee compensation charge in the fourth quarter, based on estimated stock valuation. It does not break the total down by company.
Buybacks have the same trap on a smaller scale. The cash flow statement shows $45,709 million paid, while the equity note shows $45,398 million of shares repurchased and retired, a figure the 10-K says includes any unsettled repurchases. Every run used the cash figure. Both Claude runs and both Gemini runs also mentioned the equity-note figure.
The longer notes added detail that I checked against the filing. The second Claude run listed $36.8 billion of pending acquisitions of Wiz and Intersect, and $58.5 billion of payments on data-centre leases that have not yet started. Both totals add up from the 10-K, which gives $32.0 billion for Wiz and $4.8 billion for Intersect, and $5.8 billion of short-term and $52.7 billion of long-term lease payments.
Why free cash flow barely moved
Asked why free cash flow grew so much less than operating cash flow, all six runs gave the same answer, and the filing’s own figures support it.
years = [2023, 2024, 2025]
fig, ax = plt.subplots(figsize=(9, 4.8))
w = 0.2
for k, (label, vals) in enumerate({
"Operating cash flow": [ocf[y] for y in years],
"Purchases of property and equipment": [capex[y] for y in years],
"Free cash flow": [fcf[y] for y in years],
}.items()):
ax.bar([y + (k - 1) * w for y in years], [v / 1e3 for v in vals], w, label=label)
ax.set_xticks(years)
ax.set_ylabel("$ billions")
ax.set_title("Alphabet cash flow, fiscal 2023 to 2025 (XBRL from the fiscal 2025 10-K)")
ax.legend(frameon=False)
plt.savefig("googl-fcf-capex.png", dpi=150, bbox_inches="tight")
print(f"Free cash flow 2024 {fcf[2024]:,.0f}, 2025 {fcf[2025]:,.0f}, change {fcf[2025] / fcf[2024] - 1:.1%}")
print(f"Operating cash flow change {ocf[2025] / ocf[2024] - 1:.1%}, capex change {capex[2025] / capex[2024] - 1:.1%}")
print(f"After SBC: cash flow line {fcf[2025] - sbc_cash_flow:,.0f}, Note 13 total {fcf[2025] - sbc_total:,.0f}")
figures = {
"questions": len(QUESTIONS), "runs": len(ANSWERS), "test_correct": int(score.values.sum()),
"test_total": int(score.size), "fcf_2025": fcf[2025] * 1e6, "fcf_2024": fcf[2024] * 1e6,
"fcf_growth": fcf[2025] / fcf[2024] - 1, "ocf_growth": ocf[2025] / ocf[2024] - 1,
"capex_growth": capex[2025] / capex[2024] - 1, "ocf_2025": ocf[2025] * 1e6, "capex_2025": capex[2025] * 1e6,
"ocf_increase": (ocf[2025] - ocf[2024]) * 1e6, "capex_increase": (capex[2025] - capex[2024]) * 1e6,
"capex_share": (capex[2025] - capex[2024]) / (ocf[2025] - ocf[2024]),
"returns": returns * 1e6, "returns_pct": returns / fcf[2025],
"buybacks_cash": year("PaymentsForRepurchaseOfCommonStock", 2025) * 1e6,
"buybacks_retired": year("StockRepurchasedAndRetiredDuringPeriodValue", 2025) * 1e6,
"dividends": year("PaymentsOfDividends", 2025) * 1e6,
"sbc_cash_flow": sbc_cash_flow * 1e6, "sbc_total": sbc_total * 1e6,
"fcf_after_sbc": (fcf[2025] - sbc_cash_flow) * 1e6, "fcf_after_sbc_total": (fcf[2025] - sbc_total) * 1e6,
"shares_m": float(key[8]), "sbc_gap": (sbc_total - sbc_cash_flow) * 1e6,
"sbc_total_2024": year("AllocatedShareBasedCompensationExpense", 2024) * 1e6,
"sbc_cash_flow_2024": year("ShareBasedCompensation", 2024) * 1e6,
}Free cash flow 2024 72,764, 2025 73,266, change 0.7%
Operating cash flow change 31.5%, capex change 74.1%
After SBC: cash flow line 48,313, Note 13 total 46,166
Operating cash flow rose 31.5% to $164.7 billion, purchases of property and equipment rose 74.1% to $91.4 billion, and the capex increase used up 98.7% of the extra operating cash. Both Claude runs and the second Gemini run also cited the July 2025 US tax changes, which the 10-K says allow immediate expensing of domestic research costs and faster depreciation. Both Claude runs noted that the filing does not put a number on their effect on operating cash flow. The free cash flow tutorial walks through the calculation in plain Python.
What can go wrong
Definitions are the main risk once extraction works. Free cash flow, stock-based compensation, buybacks and share counts each have more than one defensible figure in a 10-K, and a model picks one unless the prompt names it. The two stock-based compensation figures put free cash flow after it $2,147 million apart. When the prompt names the line, as test 1 did with “as shown in the cash flow statement”, all three models follow it.
Repeat runs differ. Two runs per model is a small sample, and GPT’s two analyst notes made different choices on the same prompt. Run a prompt more than once before trusting it, and keep the answers.
File format matters, and I tested only one. The models read clean text with table rows on single lines. A PDF upload to a chat app can split a table’s cells across lines, and I have no saved run showing how each app handles that.
The prompts gave the models a way out of the traps. Every prompt here asked for “NOT IN DOCUMENT” when the file had no answer, and every run used it. I did not run the prompts without that instruction, so these results say nothing about how the models behave without it.
Context files can change the run. The tools read CLAUDE.md and AGENTS.md files without announcing it. Record which ones sit above the working folder, and which skills a tool can load, along with the tool version and the model. Model names change too. The Gemini variant I planned to use disappeared before the test.
Check the numbers in code. The XBRL comparison above takes one request to companyfacts, within the SEC’s fair access rules, which ask for a declared User-Agent and at most 10 requests per second. The Claude Code DCF test used the same check on a full model, and the earlier look at LLMs like Gemini and ChatGPT for stock research covers the chat apps.
What to watch
Alphabet filed its fiscal 2024 and fiscal 2025 10-Ks on 5 February of the following year, according to its EDGAR filing index. The fiscal 2026 report will be the next chance to run the same prompts on a new filing, and it will show whether the stock-based compensation gap persists. In 2024 the two figures were close, a Note 13 total of $22,800 million against $22,785 million in the cash flow statement. The 2026 capex question has no answer in this 10-K, so any model that gives one is reading something else. New model versions arrive often in all three tools, and the version line below is what these results apply to.
My longer company research is published on the newsletter Sector Playbook, which is available by email subscription.
Sources
Alphabet’s 10-K for fiscal 2025 and its XBRL company facts on SEC EDGAR; the SEC’s EDGAR API documentation and fair access rules; the documentation for Claude Code non-interactive mode and CLAUDE.md files, Codex non-interactive mode and AGENTS.md, and OpenCode’s CLI and rules files.
Tested 3 November 2026 with Claude Code 2.1.314 (claude-opus-5-5), Codex CLI 0.166.0 (gpt-6.1-sol) and OpenCode 2.2.4 (gemini-4-argon); checking code run with Python 3.12.14, matplotlib 3.11.2, pandas 3.1.0, requests 2.34.2.
Disclaimer
This article is general information about a listed company, written for educational purposes. It is not investment advice and not a personal recommendation to buy, sell or hold any security, and it does not take into account anyone’s financial situation or objectives. I am not a registered investment adviser or a licensed financial services provider. Figures come from company filings and the other sources linked in the text, as of the dates given, and may have changed since. I may hold positions in the securities discussed; see the disclosure above. Investment decisions call for independent research or the advice of a licensed professional.
This is my own research, done with the help of AI tools.
