locally on my Mac Studio M1 with 64GB of RAM.
9 tokens per second is not fast, but acceptable.
A 15-second wait for the first token output is very good.
Popular Recent Posts
-
Choice D Since we are currently renting, we started looking at the houses we could afford. This place fits our budget, but the baby blue col... -
Sometimes the AI chats surprise me on a new level. Here is an interaction I just had when fixing my AIKO app's Human-AI Interaction (HAi... -
I never thought that I will need agent_HOA_rules.py but it is very useful! The bylaws are many and long and not text searchable, getting a ... -
I tested to belong to Haplogroup R1b1b2a1a1d1. Subclade R1b1b2a1a1d1* (as named by 23andMe ) or R1b1a2a1a1a4 (per FTDNA ) is a paternal (... -
I have run a quick test on a few LLM models I have installed locally on Mac OS with 64 GB of RAM. The test was conducted in English, but it ... -
I decided to write down a few thoughts to clarify my obsessions with creating the "multitude" of AI agents that rely on the privat... -
In my journey building software and managing technology teams, I've often witnessed the allure and danger of what Melissa Perri aptly na... -
When I work with ChatGPT, I don't just ask for an answer and move on. Instead, we engage in a real dialogue, going back and forth like ... -
In my LM Studio, I have been using the 12 billion and 24 billion parameter models on my relatively inexpensive Mac Studio M1, which has 64 G...
Most Popular Articles
-
In my journey building software and managing technology teams, I've often witnessed the allure and danger of what Melissa Perri aptly na...
-
I have noticed a very unsettling statistic on my blog. This prompted a fascinating question about AI, blogs' future, and maybe even the... -
Choice D Since we are currently renting, we started looking at the houses we could afford. This place fits our budget, but the baby blue col...
-
Prompt: What do you really see in the selfie of myself? AI: I see a volcano about to blow up and I see a lost, scared boy in front of it. ... -
Every couple of years, I write a post about my career in which I review the past and consider pivots for the future. It is time to post an u... -
I tested to belong to Haplogroup R1b1b2a1a1d1. Subclade R1b1b2a1a1d1* (as named by 23andMe ) or R1b1a2a1a1a4 (per FTDNA ) is a paternal (...
-
How to get a model from HuggingFace on Mac OS This guide documents the steps needed to download HuggingFace models (especially MLX models) c... -
Something subtle but powerful just landed in my AI pipeline: agent_Observer. It listens. Not to spy, but to sense the mood and intent. It ... -
Sometimes the AI chats surprise me on a new level. Here is an interaction I just had when fixing my AIKO app's Human-AI Interaction (HAi...
-
I have been reading a book, "Brave New Words," by Salman Khan of Khan Academy, and I got inspired to design an AI tutor (named Mai...
No comments:
Post a Comment
Please be polite.