I've been spending the past few days learning Machine Learning, Neural Nets and AI stuff.

And this's the first properly working project I did. It's basically equivalent to GPT-2 but horribly optimized and specificly training on one material.

So I wrote this neural net, line by line, understanding it all, and finally trained it on all of Shakespeare's works to generate infinitely more Shakespeare lines.

#AI #NeuralNets #MyProjects #GPT
@Dagmawi_Babi

And finally based on what I learned so far, I changed up some parameters and trained the neural network on the King James Version of the Bible.

Unlike the Shakespeare model which uses 32 blocks of characters to predict the next character in the sequence. This uses 100 characters to predict the next character, and at the same time doing 50 parallel trainings at the same time instead of 16.

It is currently training like this for 10,000 times to hopefully predict a 500 character long "bible verse".

I tried running this model on my PC but took forever. So I'm running it in the cloud and it's still taking so long, almost an hour and it's only 3000 training rounds in.

Will update you on what it generates.

#AI #NeuralNets
@Dagmawi_Babi

Update on training my neural net on the King James Version of The Bible.

Training at 4000th round.
100 characters at once,
50 sets of 100 characters in parallel.

Now I'm just worried that I didn't give it more training rounds since the content is just so much more than just Shakespeare's works. To compensate for that I did try to make the context 100 characters long. So we'll see.

So I'm just scared that it might not generate pretty specific and mind blowing results. But I could be wrong.

#AI #NeuralNets
@Dagmawi_Babi

This is the result of my neural net training on The King James Version of The Bible. 😅🤯

#AI #NeuralNets
@Dagmawi_Babi

Also about the 100,000 rounds of training i set for the KJVBibleAI, Woke up to find out that Google's Collab stopped the training.

So redoing it again.

#AI #NeuralNets
@Dagmawi_Babi

So Google Colab has been running so slow to train my neural net model. Been taking forever and it's also been shutting down and what not.

So I just left it alone and just spent the day learning about other ML concepts.

And it was just now while I was browsing inside Google Colab that I found that I could've used a GPU or a TPU.🤦‍♂

So now I'm watching it train 10k rounds in like 10 minutes which took about 40 minutes on CPU.

Well I guess I can finally train my model at 100k rounds and 1000 token blocks.

#AI #NeuralNets
@Dagmawi_Babi

Training an AI model overnight 😴

#AI #NeuralNets
@Dagmawi_Babi

So you browse through the training log and see it form sentences at each interval. And then sound more biblical. And more specifically KJV.

#AI #NeuralNets
@Dagmawi_Babi

So you can browse through the training log and see it form sentences at each interval. And then sound more sensible and then become more News-Like

#AI #NeuralNets
@Dagmawi_Babi