How and why we built Mind š§ , an open source AI Youtuber
Hey, Iām Peterā¦
Iām a physics engineer undergrad and self-taught developer, I study physics for love to science and programming to make reality some crazy ideas Iāve along this journey. I love dogs and my favorite spiderman is Tobey Maguire ;)
One of my goals as child was to become a Youtuber, Iāve uploaded videos to almost 7 channels since I was 12. So I know how is it like to try to record -> edit -> write -> upload for each video. This process could take from 1 hour to days.
Thatās why Iāve been working on TurboReel a couple months now, a Video Engine powered by AI. Iāll tell you more in a minute.
TurboReel was great, it already had a Redit Engine and a Script Engine, which allows you to automaticly create those Minecraft parkour videos. However, after gathering feedback I came to the conclusion that it wasnāt enough and something more powerful was needed in order to truly change the way we see video creation.
I knew I had to do something about it. So, on the night of October 22, I stayed up all night coding a < JSON to Video > module using MoviePy to process videos. After it, the first thought that popped into my head was to have an LLM generate these JSONs that could become basicly any kind of video and format.
And thatās how Mind, The AI Youtuber, was born. Itās still a baby, but itās growing fast.
What the Heck is Mind?
Mind it is an Experimental Content Creator or Video Editor, part of the open source TurboReelās Video Engine toolkit. Mind felt natural to persue after building the JSON-to-Video parser.
Recently, Iāve started categorizing all tasks into two types: LLM-automatable and non-LLM-automatable.
Video creation is an LLM-automatable task, and LLMs could become highly effective at it, thanks to the JSON format and certain techniques.
Letās take a brief look at what Mind could do:
Mind can generate basic videos for any concept you can think of. The JSON is divided into Script, Videos, Images, Audio, and Text, with Script typically being the most important part. Mind synchronizes the other elements based on the Script, though this could be adjusted in future versions for added flexibility.
Whatās the Tech Stack?
TurboReelās Flowchart
Mind and TurboReel code is available on github, so you can check out the code, fork it, learn from it, make your own, submit a PR (Iād love you forever if you did š)ā¦ whatever!
It is built using MoviePy, a Python library. And the AI features are primarly achieve with the OpenAI API, this includes: Generating scripts, searching for images, syncing assets, generating voice, summarizing scripts in certain scenarios, etc.
In order to save some bucks, TurboReel uses stock images from the Pexels API. This could be easly done with an AI generator for images, and should be implemented very soon.
Timeline
Origin
This year, I decided to start some side gigs, one of which was selling POD (print-on-demand) merch. My goal was to automate the entire video creation process and use it as a marketing channel for the merch.
TL;DR: POD didnāt work out.
During that process, I researched options to automate video creation, like faceless video platforms, InVideo, and others. These platforms are okay, but they donāt feel naturalāvery stock and corporate. Iām not even sure how to describe it.
I stumbled upon a couple of github repos that got me thinking:
- RedditVideoMakerBot (Link): This was built specifically for Reddit videos.
- ShortGPT (Link): A super cool project with some awesome features, like adding relevant images to the video script automatically. I was blown away by what it had achieved.
I couldnāt quite get them to work smoothly. And also, I tend to enjoy building things from scratch (classic coder behavior), so I decided to build an MVP
Testing the idea
After about 10 days of work, I had a working prototype. This was the first video generated correctly:
Next thing I did was some spam on Reddit,
maybe your are seeing this because of that lol
Reactions wereā¦ mixed, to say the least.
Some people were like, āmore trash content on TikTok?ā, but others were excited and said, āIāve been waiting for something like this to get back into YouTube.ā I decided to hear the positive vibes
I posted a waitlist (see link in the resources section) that 60 people signed up, got almost 30 stars on Github and 14 people signed to the beta trough a form (see link in the resources section). All this in 1 week.
People wanted it :) and I was very excited about the idea.
TurboReelās Launch
Besides the open source repo, I worked on TurboReel platform to make it accessible for non-technical people. Shout out to Wasp, āThe fastest way to develop full-stack web apps with React & Node.js..ā
Their OpenSaaS template is great.
On the server side
Working with FFmpeg was already a headache, but deploying this kind of app on the cloud was a whole other experience. Not to mention AWS servers costs.
I had just moved to a new house where the internet was only 5 Mbps! I had to get a better connection (now Iām at 500 Mbps, thankfully). The docker builds and pushes were a nightmare. I used AWSās free EC2 tier to run the builds and push them to ECS, but each build took 5 minutes, and I had to debug it 20 times. So, thatās like 100 minutes of just deploying! But hey, I learned Docker and ECS in the process, so it wasnāt all bad.
By mid-October, Closed Beta was up.
Feedback Time
The real uniqueness of any product comes from the users. Iāve been super focused on gathering feedback from everyone whoās tried it. Feel free too do so!
A YC friend once told me, āPeople say they want something, but itās your job to figure out the real problem behind what theyāre asking for.ā
A lot of users asked for more formats and personalization options(captions, assets, etc.), which gave me the direction for what became Mind.
Iāve been reaching out directly to users and the Discord server is has been very helpful on that too.
Something is missingā¦
Even though TurboReelās Reddit and Script Engines were cool and powerful, something still felt off. I realized it was missing one big thing: Freedom.
At first, I thought about building a specific engine for each scenario (documentaries, product showcases, education, etc.). But that seemed too limited and very time consuming. TurboReel aims to change how we generate videos that feel real and human, not just fit into a few categories
So, I asked myself: Whatās a format any program can read and is pretty universal? JSON. After doing some research, I couldnāt find any MoviePy-based projects using JSON. This surprised me because it seemed like such a necessary feature ā even without LLMs
worked on it, and here we are :))
Lets say Mind is the combination of all previous couple months work + LLMs, which make this problem even more interesting.
Future Plans
JSON parser is almost ready.
At the moment, this are the most important things to do:
- Improve how the AI agent handles the JSON generation and understand the different relations trough it.
- Add several verification layers to the JSON-Video parser, that would help the AI agent to have kind of a Reinforcement Learning approach to generate better JSONs.
- Even thought assets like videos and audios are capable of being processed by the parser, this is based on paths(strings) and not actual assets. So it provides a bad user experience, ideal scenario would be to let the user input actual assets or generate them with AI.
- Try more models, like Claude, Mistral, Gemini, etc.
After that necessary changes, sky is the limit. You could add whatever your mind can think of. Such as:
- AI avatars.
- Allow users to interact with existing videos through text commands.
- Design JSON templates for industries like marketing, education, etc.
- Create cool animations.
- Build a worldwide gallery for people to share and showcase their video templates.
Final Words + More Resources
My intention with this article was to help others who might be considering launching their own open source project or side-project, also get some people interested in the project. So I hope thatās been the case here. If you still have any questions, donāt hesitate to ask. Reach me on X or Discord.
Here are also the most important links from this article along with some further resources:
- šØāš» TurboReel GitHub Repo
- š¤ Mind - Early Access
- š¬ TR Discord
- āļø Initial TurboReel Reddit Post
- š TR Waitlist and TR Beta form
- š„ TR Tiktok Channel where Iāve been Testing Engagement
- š Wasp - Free Open SaaS Template
Oh, and if you found these resources interesting, donāt forget to support TurboReel by starring the repo on GitHub! āļø
