Everything pertaining to the technological singularity and related topics, e.g. AI, human enhancement, etc.
![](http://web.archive.org./web/20240524021501im_/https://preview.redd.it/well-done-u-fucksmith-v0-ozszjopfr62d1.jpeg?width=640&crop=smart&auto=webp&s=7355bb2c9140d0c681c8f741688169aebac2b48f)
![r/singularity - Well done, u/fucksmith](http://web.archive.org./web/20240524021501im_/https://preview.redd.it/well-done-u-fucksmith-v0-ozszjopfr62d1.jpeg?width=640&crop=smart&auto=webp&s=7355bb2c9140d0c681c8f741688169aebac2b48f)
Everything pertaining to the technological singularity and related topics, e.g. AI, human enhancement, etc.
Character.AI lets you create and talk to advanced AI - language tutors, text adventure games, life advice, brainstorming and much more.
For me its "Grandpadawan" XD
OpenAI is an AI research and deployment company. OpenAI's mission is to ensure that artificial general intelligence benefits all of humanity. We are an unofficial community. OpenAI makes ChatGPT, GPT-4, and DALL·E 3.
ChatGPT is about to be turned into yet another propaganda tool by News Corp. just in time for the elections. This is a disgusting and disappointing move by Open AI. News Corp themselves have pleaded several times that no person in their right mind would believe they are actually news, but this is what they are going to use as journalism? Please voice your concern by unsubscribing from their service and finding another tool. Make sure you specify why you unsubscribed.
To cancel your ChatGPT Plus subscription, follow these steps:
Log into ChatGPT: Visit the ChatGPT website and log in with your credentials.
Access Subscription Settings: Click on "My Plan" in the left sidebar.
Manage Subscription: In the pop-up window, click "Manage my subscription."
Cancel Plan: On the checkout page, select "Cancel Plan." Your cancellation will take effect the day after your next billing date. To avoid being charged for the next billing period, ensure you cancel at least 24 hours before your next billing date.
Welcome to r/aiArt ! A community focused on the generation and use of visual, digital art using AI assistants such as Wombo Dream, Starryai, NightCafe, Midjourney, Stable Diffusion, and more.
Everything pertaining to the technological singularity and related topics, e.g. AI, human enhancement, etc.
Character.AI lets you create and talk to advanced AI - language tutors, text adventure games, life advice, brainstorming and much more.
/r/StableDiffusion is back open after the protest of Reddit killing open API access, which will bankrupt app developers, hamper moderation, and exclude blind users from the site. More info: https://rtech.support/docs/meta/blackout.html#what-is-going-on Discord: https://discord.gg/4WbTj8YskM Check out our new Lemmy instance: https://lemmy.dbzer0.com/c/stable_diffusion
OpenAI is an AI research and deployment company. OpenAI's mission is to ensure that artificial general intelligence benefits all of humanity. We are an unofficial community. OpenAI makes ChatGPT, GPT-4, and DALL·E 3.
Subreddit to discuss about Llama, the large language model created by Meta AI.
The goal of the r/ArtificialIntelligence is to provide a gateway to the many different facets of the Artificial Intelligence community, and to promote discussion relating to the ideas and concepts that we know of as AI. These could include philosophical and social questions, art and design, technical papers, machine learning, where to find resources and tools, how to develop AI/ML projects, AI in business, how AI is affecting our lives, what the future may hold, and many other topics. Welcome.
I regularly find myself saying things like "Can you ...." or "Do it again for this please". Are you polite, neutral, or rude to AI?
Subreddit to discuss about ChatGPT and AI. Not affiliated with OpenAI. Thanks Nat!
I asked my ChatGPT to pick a name that it liked for itself. It picked Echo. Does it say the same thing for everybody?
Everything pertaining to the technological singularity and related topics, e.g. AI, human enhancement, etc.
Subreddit to discuss about Llama, the large language model created by Meta AI.
Welcome to the Janitor AI sub! https://janitorai.com https://discord.gg/janitorai
Subreddit to discuss about Llama, the large language model created by Meta AI.
A few days ago, rgerganov's RPC code was merged into llama.cpp and the old MPI code has been removed. So llama.cpp supports working distributed inference now. You can run a model across more than 1 machine. It's a work in progress and has limitations. It currently is limited to FP16, no quant support yet. Also, I couldn't get it to work with Vulkan. But considering those limitations, it works pretty well. Inference is limited by network bandwidth. Using a 1 gigabit ethernet connection is faster than using a slower wifi connection. And the overall speed seems to be limited by the slowest machine. See my numbers below.
You can read more about it here.
Here are some numbers between a M1 Max Studio and a PC with a 7900xtx. The model is Tiny Llama FP16.
This first set of numbers is from the Mac as the client.
Mac only
llama_print_timings: prompt eval time = 199.23 ms / 508 tokens ( 0.39 ms per token, 2549.77 tokens per second) llama_print_timings: eval time = 8423.24 ms / 511 runs ( 16.48 ms per token, 60.67 tokens per second)
7900xtx only
llama_print_timings: prompt eval time = 100.50 ms / 508 tokens ( 0.20 ms per token, 5054.98 tokens per second) llama_print_timings: eval time = 10574.48 ms / 511 runs ( 20.69 ms per token, 48.32 tokens per second)
Mac + 7900xtx
llama_print_timings: prompt eval time = 230.29 ms / 508 tokens ( 0.45 ms per token, 2205.92 tokens per second) llama_print_timings: eval time = 11147.19 ms / 511 runs ( 21.81 ms per token, 45.84 tokens per second)
Here are numbers from the 7900xtx PC as the client.
Mac only
llama_print_timings: prompt eval time = 253.78 ms / 508 tokens ( 0.50 ms per token, 2001.77 tokens per second) llama_print_timings: eval time = 10627.55 ms / 511 runs ( 20.80 ms per token, 48.08 tokens per second)
7900xtx only
llama_print_timings: prompt eval time = 40.93 ms / 508 tokens ( 0.08 ms per token, 12412.34 tokens per second) llama_print_timings: eval time = 4249.10 ms / 511 runs ( 8.32 ms per token, 120.26 tokens per second)
Mac + 7900xtx
llama_print_timings: prompt eval time = 198.44 ms / 508 tokens ( 0.39 ms per token, 2559.98 tokens per second) llama_print_timings: eval time = 11117.95 ms / 511 runs ( 21.76 ms per token, 45.96 tokens per second)
As you can see, the inference overall seems to be limited by the speed of the network connection. Which is about 46t/s for this model. Even though both the Mac and the 7900xtx are faster than 48t/s locally, they are limited to 48t/s when run remotely.
To further illustrate that the network is the bottleneck, here's the numbers for the Mac running over wifi instead of ethernet.
llama_print_timings: prompt eval time = 737.93 ms / 508 tokens ( 1.45 ms per token, 688.41 tokens per second) llama_print_timings: eval time = 42125.17 ms / 511 runs ( 82.44 ms per token, 12.13 tokens per second)
It's only 12t/s for TG versus 48t/s.
One last number for number sake. Here's the llama 3 7B model at FP16 running across both.
llama_print_timings: prompt eval time = 826.07 ms / 508 tokens ( 1.63 ms per token, 614.96 tokens per second) llama_print_timings: eval time = 29902.27 ms / 511 runs ( 58.52 ms per token, 17.09 tokens per second)
Subreddit to discuss about ChatGPT and AI. Not affiliated with OpenAI. Thanks Nat!
Everything pertaining to the technological singularity and related topics, e.g. AI, human enhancement, etc.
Subreddit to discuss about ChatGPT and AI. Not affiliated with OpenAI. Thanks Nat!
Welcome to the Janitor AI sub! https://janitorai.com https://discord.gg/janitorai
Character.AI lets you create and talk to advanced AI - language tutors, text adventure games, life advice, brainstorming and much more.