As usual, the Chrome team is leading the charge on some exciting new web platform tech. The goal is to release some prototypes and eventually write up the feature as a browser standard that would make its way into all browsers (i.e. not just Chrome).

The point is, it’d run completely on-device (no cloud access, works offline), so it’d be a very small model, but would likely still be smart enough for a lot of tasks - e.g. summarizing text, converting a list of words into a grammatically correct sentence/description, guessing an appropriate emotion based on some character dialogue, etc.

Article: https://developer.chrome.com/docs/ai/built-in

The key problem with these text generation models is how massive they are. They’re so big that they could literally fill your entire device (for smart phones and cheap laptops, at least), and would bloat the initial browser download time from a few minutes to a few days for a lot of people.

Still, smaller models are getting surprisingly smart, and while they’re still several times the size of the actual browser download itself, this download can be done in the background.

Either way, I’m excited about this new direction, because there are lots of tasks that don’t require an extremely smart model, and so it’s overkill to use /ai-text-plugin, especially since it means ads will be shown for non-logged-in users.

One problem that I do anticipate, is that the models will be extremely “safety-oriented”, meaning refusal to even generate stuff like violence in a DnD fantasy adventure, and stuff like that. I know from experience that Google’s Gemini models have false-positive-refusal rates that almost make them unusable even for many sfw tasks. There is a mention of LoRA fine-tuning in the article, which is very exciting and might help with that. If you’re a web dev, you can use the links on the page to test their prototypes and give constructive+professional feedback on them. It’d be good for the health of the web platform to have some of the feedback be for use-cases like Perchance, and not just e.g. business applications.

Tangentially, builders here may also be interested in Transformers.js which allows you to run AI models in your browser. Ad-free AI plugins could already be created using this project, although for a lot of models the download times are a bit too long, and processing times also a bit too long (for mobile devices especially). Still, the situation is improving quite rapidly. /ai-character-chat already uses Transformers.js for text embedding.