This post concerns the appendix (in its standalone version, written by me) to Lena Ji and Fumiaki Suzuki's paper on motivic classes of Fano schemes of lines.

A few years ago we wrote a paper on decompositions of Fano schemes of linear subspaces on intersections of two quadrics. For $g\ge 2$ and $0\le k\le g-2$, consider a smooth intersection $Q_1\cap Q_2\subset\mathbb{P}^{2g+1}$ with associated hyperelliptic curve $C$. One of our conjectures predicts the following identity in $\mathrm{K}_0(\mathrm{Var})$: \begin{equation} [\mathrm{F}_k(Q_1\cap Q_2)] = \sum_{i=0}^{k+1}\mathrm{M}_{g,k,i}(\mathbb{L})[\operatorname{Sym}^i C], \end{equation} where

  • $\mathbb{L}=[\mathbb{A}^1]$,
  • $\operatorname{Sym}^i C$ is the $i$th symmetric power of $C$, and
  • for $0\le i\le k+1$, the coefficient is \begin{equation} \begin{aligned} \mathrm{M}_{g,k,i}(\mathbb{L}) ={}&\mathbb{L}^{i(g-k-1)}\Biggl( \binom{2g-k-i}{k+1-i}_{\mathbb{L}}\\ &-\bigl(\mathbb{L}^{g-k-1}+\mathbb{L}^{g+2k-3i}\bigr) \binom{2g-k-i-4}{k-i}_{\mathbb{L}}\\ &-\bigl( \mathbb{L}^{g-k}+\mathbb{L}^{g-i}+\mathbb{L}^{g+k-2i} +\mathbb{L}^{3g-3k-4}+\mathbb{L}^{3g-2k-i-4}+\mathbb{L}^{3g-k-2i-4} \bigr)\binom{2g-k-i-4}{k-i-1}_{\mathbb{L}}\\ &-\bigl( \mathbb{L}^{3(g-k-1)}+\mathbb{L}^{3(g-k-1)+1} +\mathbb{L}^{3g-2k-i-3}+\mathbb{L}^{3g-2k-i-2} \bigr)\binom{2g-k-i-4}{k-i-2}_{\mathbb{L}}\\ &-\mathbb{L}^{4(g-k)-2} \binom{2g-k-i-4}{k-i-3}_{\mathbb{L}} \Biggr). \end{aligned} \end{equation}
Here the subscript $\mathbb{L}$ denotes a Gaussian binomial coefficient, which is zero when its lower index is negative or exceeds the upper index.

Important for today's post is that we also conjectured that these polynomials are effective: their coefficients are nonnegative.

Last year, IMProofBench was looking for interesting benchmark problems (oh, the times have changed) and I submitted this effectivity question as something that seemed to require expertise beyond what I had in store. I did try to solve it myself for some time, but none of my (admittedly limited) bag of tricks seemed to work, a fact related to me not being a combinatorialist.

The first attempts by LLMs as part of IMProofBench were utter nonsense. However, back in March 2026, one LLM managed to produce what looked like a correct proof (reproduced below), using an interesting new ingredient! When I finally had a look at it in June, I wrote up a more human-readable version of the proof, at the very least for my own sake to verify the argument. I shared it with a few people back then, but didn't settle on what to do with it.

However, when Lena (and Fumiaki) reached out about a preprint they had been working on which proves our full conjecture for $k=1$, we agreed that it would be good if the LLM-proof of the effectivity conjecture became an appendix, so that others can use the effectivity part. I rewrote the existing writeup, making sure that its exposition was as good as I could make it, and it is now available as an appendix (the link is to an independent PDF) to Lena Ji and Fumiaki Suzuki's paper on motivic classes of Fano schemes of lines. The appendix proves effectivity for all the polynomials in the conjecture; the motivic identity itself remains conjectural in general.

The original answer

For full disclosure, here is the original LLM-proof of the conjecture. I can imagine that for some mathematicians, this abbreviated and terse proof is enough. However, if the goal is that I (and maybe other non-combinatorialists) understand what's going on, I do think (and hope) that my writeup is more useful.

Let $a=g-k-1\ge 1$ and $m=k-i\in\{-1,0,\dots,k\}$. The polynomial from the conjecture can then be written as \begin{equation} M_{g,k,i}(t)=t^{ia}B_{a,m}(t), \end{equation} where $B_{a,m}(t)$ is the following expression in $a$ and $m$: \begin{equation} \begin{aligned} B_{a,m}(t)={}&\binom{2a+m+2}{m+1}_t -(t^a+t^{a+1+3m})\binom{2a+m-2}{m}_t\\ &-(t^{a+1}+t^{a+1+m}+t^{a+1+2m}+t^{3a-1}+t^{3a-1+m}+t^{3a-1+2m})\binom{2a+m-2}{m-1}_t\\ &-(t^{3a}+t^{3a+1}+t^{3a+m}+t^{3a+m+1})\binom{2a+m-2}{m-2}_t -t^{4a+2}\binom{2a+m-2}{m-3}_t. \end{aligned} \end{equation}

Here $\binom{n}{r}_t$ is the Gaussian binomial coefficient, taken to be zero for $r<0$ or $r>n$. It is enough to prove that $B_{a,m}(t)\in\mathbb{Z}_{\ge 0}[t]$ for all $a\ge 1$ and $m\ge -1$.

1. Generating function of $B_{a,m}$

Write \begin{equation} (x;t)_n:=\prod_{r=0}^{n-1}(1-t^r x). \end{equation}

We use the standard Gaussian binomial generating series \begin{equation} \sum_{r\ge 0}\binom{n+r}{r}_t y^r=\frac1{(y;t)_{n+1}} \qquad (n\ge 0). \end{equation}

Define \begin{equation} F_a(x):=\sum_{m\ge -1} B_{a,m}(t)x^{m+1}. \end{equation}

Applying the above identity term by term gives \begin{equation} \begin{aligned} F_a(x)={}&\frac1{(x;t)_{2a+2}} -\frac{t^a x}{(x;t)_{2a-1}} -\frac{t^{a+1}x}{(t^3x;t)_{2a-1}}\\ &-x^2\left( \frac{t^{a+1}+t^{3a-1}}{(x;t)_{2a}} +\frac{t^{a+2}+t^{3a}}{(tx;t)_{2a}} +\frac{t^{a+3}+t^{3a+1}}{(t^2x;t)_{2a}} \right)\\ &-x^3\left( \frac{t^{3a}+t^{3a+1}}{(x;t)_{2a+1}} +\frac{t^{3a+2}+t^{3a+3}}{(tx;t)_{2a+1}} \right) -\frac{t^{4a+2}x^4}{(x;t)_{2a+2}}. \end{aligned} \end{equation}

Now multiply by $(x;t)_{2a+2}$. Using the quotient identities \begin{equation} \frac{(x;t)_{2a+2}}{(x;t)_{2a-1}}=(1-t^{2a-1}x)(1-t^{2a}x)(1-t^{2a+1}x), \end{equation} \begin{equation} \frac{(x;t)_{2a+2}}{(t^3x;t)_{2a-1}}=(1-x)(1-tx)(1-t^2x), \end{equation} \begin{equation} \frac{(x;t)_{2a+2}}{(x;t)_{2a}}=(1-t^{2a}x)(1-t^{2a+1}x), \quad \frac{(x;t)_{2a+2}}{(tx;t)_{2a}}=(1-x)(1-t^{2a+1}x), \end{equation} \begin{equation} \frac{(x;t)_{2a+2}}{(t^2x;t)_{2a}}=(1-x)(1-tx), \quad \frac{(x;t)_{2a+2}}{(x;t)_{2a+1}}=1-t^{2a+1}x, \quad \frac{(x;t)_{2a+2}}{(tx;t)_{2a+1}}=1-x, \end{equation}

and collecting coefficients of $x^0,x^1,x^2,x^3,x^4$, one gets \begin{equation} (x;t)_{2a+2}F_a(x) =(1-t^a x)(1-t^{a+1}x)(1-t^{2a+1}x^2). \end{equation}

Hence \begin{equation} F_a(x) =\frac{(1-t^a x)(1-t^{a+1}x)(1-t^{2a+1}x^2)}{(x;t)_{2a+2}} =\frac{1-t^{2a+1}x^2}{(x;t)_a\,(t^{a+2}x;t)_a}. \end{equation}

So we have proved the compact identity \begin{equation} \sum_{m\ge -1} B_{a,m}(t)x^{m+1} =\frac{1-t^{2a+1}x^2}{(x;t)_a\,(t^{a+2}x;t)_a}. \end{equation}

2. Manifestly effective form

Now use the elementary decomposition \begin{equation} 1-t^{2a+1}x^2=(1-t^{a-1}x)+t^{a-1}x(1-t^{a+2}x). \end{equation}

Substituting this into the previous formula gives \begin{equation} F_a(x) =\frac{1}{(x;t)_{a-1}(t^{a+2}x;t)_a} +\frac{t^{a-1}x}{(x;t)_a(t^{a+3}x;t)_{a-1}}. \end{equation}

This is manifestly positive, because every factor \begin{equation} \frac1{1-t^j x}=\sum_{r\ge 0} t^{jr}x^r \end{equation}

has coefficients in $\mathbb{Z}_{\ge 0}[t]$, so both summands lie in $\mathbb{Z}_{\ge 0}[t][[x]]$. Therefore every coefficient of $F_a(x)$, and hence every $B_{a,m}(t)$, lies in $\mathbb{Z}_{\ge 0}[t]$. Since $M_{g,k,i}(t)=t^{ia}B_{a,m}(t)$ is a monomial multiple, its coefficients are nonnegative as well.

3. The final formula in the original variables

Returning to $a=g-k-1$ and $m=k-i$, we obtain \begin{equation} \begin{aligned} M_{g,k,i}(t) =t^{i(g-k-1)}[x^{k+1-i}]\Biggl(& \prod_{j=0}^{g-k-3}\frac1{1-t^j x} \prod_{j=g-k+1}^{2g-2k-1}\frac1{1-t^j x}\\ &+t^{g-k-2}x \prod_{j=0}^{g-k-2}\frac1{1-t^j x} \prod_{j=g-k+2}^{2g-2k-1}\frac1{1-t^j x} \Biggr), \end{aligned} \end{equation}

An empty product equals $1$. The formula makes effectivity explicit.

Equivalently, if $h_r$ denotes the complete homogeneous symmetric polynomial, then \begin{equation} \begin{aligned} M_{g,k,i}(t)=t^{i(g-k-1)}\Bigl( &h_{k+1-i}(1,t,\dots,t^{g-k-3},t^{g-k+1},\dots,t^{2g-2k-1})\\ &+t^{g-k-2} h_{k-i}(1,t,\dots,t^{g-k-2},t^{g-k+2},\dots,t^{2g-2k-1}) \Bigr), \end{aligned} \end{equation}

Empty ranges are omitted, and $h_r=0$ for $r<0$. Each $h_r$ is a sum of monomials with nonnegative integer coefficients. Thus $M_{g,k,i}(t)$ is effective for all $g\ge 2$,  $0\le k\le g-2$, and $0\le i\le k+1$.