As LLMs and especially coding agents became part of our daily workflows, we started relying more on actual judgement of models we use — judgement, to me, is any decision they default to based on their training and pre-processing without humans actually directing them with prompts or skills.

It might be a simple decision about using tabs or spaces, or a more complex one about how to structure your tests and test fixtures, but you can already get a lot done by offloading decision-making as well.

While this is fast improving, I wanted to test my understanding by asking them to do an extremely common test by Simon Willison of generating a vector image of a pelican riding a bicycle, but with a twist: instead of asking for an SVG, ask for a conceptually different vector format PostScript (really, a programming language), which is closer to the plotting/printing model from 80s, with instructions like moveto, Y-axis actually going upwards, for loops, matrix transformations etc — enough difference in semantic interpretation and less training data to throw a frontier model off :)

My expectation was that they would be a generation or two behind compared to what Simon is getting with SVGs.

I'll let you be the judge if my foresight was right or not.

Prompt: Generate a native PostScript image of a pelican riding a bicycle.

Some quick notes

• I tried converting PostScript files with ImageMagick first, but after seeing it miss a few things during conversion, decided to use GhostScript loader from within Inkscape instead and export a PNG out.
• Gemini 3.7 Flash had a broken first version which it was able to fix with the Ghostscript error message passed back (I think it was wrong parameters for ellipse command).
• Opus 5 took around 4 iterations of rendering to a raster image and reviewing (also hitting a PS syntax error), but managed to improve with each cycle. I did not check the costs, but it was probably expensive :)
• My first few attempts were with a slightly different prompt of "Make an PS or EPS (PostScript) of a pelican riding a bicycle" ("an" typo included — I probably went for EPS first and never corrected the language). This resulted in greyscale images in ChatGPT confirming that this might be worth looking into. I did not approach it scientifically at all, but switching to "Generate" to match Simon's version always produced a colorized version.
• All of the text in this post was hand-typed, including the em-dashes.