We introduce clever, the first curated benchmark for evaluating the generation of specifications and formally verified code in lean.The benchmark comprises of 161 programming problems;It evaluates …While, as we mentioned earlier, there can be thorny “clever hans” issues about humans prompting llms, an automated verifier mechanically backprompting the llm doesn’t suffer from these.