Today a friend sent me this podcast that is entirely generated by artificial intelligence.
It's just insane how AI and ML is getting advanced. You can't almost tell if it's real or not. It's insane. This episode is "Joe Rogan Interviews Steve Jobs"
Huge day indeed for AI and LLMs, congrats to Meta. This is now the most capable LLM available directly as weights to anyone! 👏
Pretrained and fine-tuned models are available with 7B, 13B and 70B parameters.
Llama 2 pretrained models are trained on 2 trillion tokens, and have double the context length than Llama 1. Its fine-tuned models have been trained on over 1 million human annotations.
Downloaded LLAMA2 and set up the 13B param one locally and it's IMPRESSIVE!!!
First off it's like GPT-3 kind of good for everyday use. Second off it didn't require hugee resources and it was actually fast! 🤯
So the first thought that came to mind was to set it all up on my server for all y'all to use. But ofc shared servers have limited resources and would run at all.
I'll experiment and see if I can adjust the memory requirements to run on my servers. Then we can all use it freely.
You should try it, specially if you have a nice gaming PC. And you should try it with the c++ version instead of python which is also super easy to set up and run • https://github.com/ggerganov/llama.cpp
I was using Andrej Karpathy's lessons when I made my GPT a couple months back. It's such a simple idea but so effective. It's just wild even when I wrote it and understood it all from scratch to think that this was possible.
Kaggle is a HUGE resource when it comes to learning machine learning and AI along side progamming. • https://www.kaggle.com/learn
Not only does it have an extensive list of datasets, AI models and machine learning related codes but it also has an epic resource of learning materials.
You get to learn, write ML code, train it on datasets, compete in competitions and so much more all in one free platform. This's mind blowing!
Open AI just announced a text to video model 🤯 • OpenAI.com/Sora
OpenAI's Sora, a text-to-video model, can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions.
Devin is marketed as the first AI software engineer, but don't let that throw you off. Though It's coding and debugging capabilities are insane, it's still has a long way to go AND will come to a stop.
It's great but I know it'll replace devs, What most people don't understand is that these models are LLMs. They're statistical predictive models that don't understand what they're doing. So just like how ChatGPT hasn't stopped all the authors of the world, this won't stop the coders too.
They released the base model weights and network architecture. It also shows that the model has 314 billion parameters and is a mixture of experts model. And it is not fine tuned alot. So chatting with it needs a bit of a config and fine tuning.
For a 314B parameter model, it is very undertrained. The benchmarks show that the model is of the GPT 3.5 level. GPT 3.5 is estimated to be a 20B parameter model.
That tone is just so unnecessary. Specially when IT plays and talks to the dog, UGH so creepy. People are so doomed. Dating this is gonna be happening soon ofcourse.
@Naklecha released a repo that implements LLAMA3 from scratch -- every matrix multiplication from attention across multiple heads, positional encoding and every other layer in between has been carefully unwrapped & explained. • https://github.com/naklecha/llama3-from-scratch
@DotNetWarrior trained this self driving AI. I thought it was pretty awesome.
The training was running ~20X speed from real-time and it was trained for 12 hours exactly, but it has gained 600hours of expirience, and the AI learnt to navigate around the map and get to it's destination pretty well. 🔥
Plus the video is pretty fun and enjoyable. Great job dude, can't wait to see more. 👏
