Mad Skills - New Claude Feature

Anthropic released a new feature called Skills and I have been messing around with them. When I first read about skills it was not clear how they help. They sounded like what I have using Projects for in ChatGPT - a workspace with it's own context (memory). What I found is that they are different, but it is possible to use a Skill from Claude in ChatGPT.

My partner and I were playing a lot of Pokemon Go recently. So it seemed logical to create a skill using Claude to help answer some of the strategy questions we were discussing with one another.  

I created three data files that 1/ rules of the game, 2/ the requirements and rewards for each level, and 3/ some gameplay strategy data. I asked Claude to generate a Skill that helped players identify actions they can take to optimize their gameplay to meet various goals. I also included some examples of good optimizations and bad ones to help. Even given what I though was a thorough prompt, Claude came back with several questions to help define the scope of the Skill. Once we were satisfied it was "good enough", it produced a few files, the primary of which was a .skill file.

I played around with the Skill and it did a reasonable job. They key to getting the Skill to run was to 1/ ensure that it was uploaded and enabled in the Skill manager, and 2/ when prompting in Claude using keywords that ensured it knew to use the skill. The keywords are something that I went back and forth with Claude on when building the skill. In the end, it felt similar to using and MCP where there was an inference / mapping that Claude has internally to know what to call / use.

I think of Skills as having a predefined list of prompts that automatically called when needed.

Out of curiosity, I took all the files that Claude output for the Skill and uploaded them into a ChatGPT project. The question I was trying to answer is whether the Project would retain the context of all the prompting / data and behave similar to the Skill. Since the .skill file is just a zip, I uploaded each of the file individually. And it did work, albeit not as well as it did in Claude. My guess is that there were two contributors to the less outcomes; the first being that they are different models and the Skill benefited from the model that created it. Second that the ChatGPT Project did worse over time as the context of the Project grew with each subsequent chat (I am assuming you know how Projects work).

My takeaway. I like Skills,especially the help I get from Claude in creating them. I wish I could have more than 10 active at any time. That said, I will likely use a combination of both Claude Skills and ChatGPT (maybe projects). Why? I don't run into any daily limits with ChatGPT. I have the Pro version of Claude and I regularly run into limits and have to way until the next day for it to reset. It's frustrating that something like this and not the quality / usefulness of Skills is the reason for this bi-model usage. 

Addendum: I pay for ChatGPT and Claude now.

Very Sparkly (Evolving Thinking on GenAI)

Tired of my GenX references yet? No? Good, here is another one.

Let me explain. As you may notice from my more recent blog entries, "AI" is something I have been frequently thinking about. Especially trying to work through the hype, where GenAI is very sparkly and attention grabbing, to a place where I see how AI makes things better. Further, what is the intersection of my years of engineering and building AI solutions? Is there one? Take for instance this entry from a year ago where this journey started to gain traction for me. Or me thinking about Vibe coding back in July.

I’ve been thinking about what “engineering” even means now that anyone can spin up an LLM and build something that looks like it works. It’s too easy to confuse “the demo runs” with “the system works.” So here’s where I’ve landed - four realities that to consider. These are not my original work (thanks Nate), but landed in my conciseness at the exact right time.

1. If you can’t write the invariant, you haven’t engineered it.
This is the difference between vibe coding and engineering. “Make it work” is not the same as “define what working means.” AI gives you plausibility, not correctness. The invariant is what keeps you from gambling with probabilities and calling it architecture.

2. If you can’t measure it in production, you didn’t build it.
Anyone can make a demo now. Real engineering starts when actual users show up and start breaking things in creative ways. Observability isn’t a bonus -it’s the only way to know if the magic still works after it leaves the notebook.

3. If you can’t explain why it failed to a regulator, you haven’t owned it.
“The AI did it” won’t fly with the FDA, SEC, or anyone with a pulse. If you can’t walk a smart non-engineer through what went wrong, you don’t understand your own system. Ownership means accountability, not just implementation.

4. Good system design still matters.
Data is still the asset. Models are temporary. The LLM you love today will be obsolete in six months. Build so you can swap the engine without rebuilding the car. The fundamentals haven’t changed - only the excuses have. The data represents your value / asset, the LLM is a processor to help you realize the value of your data.

Bottom line: these form a lifecycle - specify, verify, and own. The tools changed. The responsibility didn’t.