Discuss Scratch
- Discussion Forums
- » Project Ideas
- » How unrealistic is a basic LLM made using neural networks in scratch
- StarWalker600
-
100+ posts
How unrealistic is a basic LLM made using neural networks in scratch
Title.
EDIT1: For those wondering whats an llm its large lanuage model eg. chatgpt, gemini, deepseek, claude, etc
EDIT2: Pls give me positive coments
EDIT1: For those wondering whats an llm its large lanuage model eg. chatgpt, gemini, deepseek, claude, etc
EDIT2: Pls give me positive coments
Last edited by StarWalker600 (Feb. 2, 2025 14:34:13)
- BigNate469
-
1000+ posts
How unrealistic is a basic LLM made using neural networks in scratch
Basically impossible- the training data alone would exceed the project.json maximum size limit.
Not to mention that without GPU acceleration it would be horribly slow.
Not to mention that without GPU acceleration it would be horribly slow.
- StarWalker600
-
100+ posts
How unrealistic is a basic LLM made using neural networks in scratch
cant we use clouds vars to connect scratch project to a external service or smth & run the neural network there i mean there are things like scratch attach Basically impossible- the training data alone would exceed the project.json maximum size limit.
Not to mention that without GPU acceleration it would be horribly slow.
- BigNate469
-
1000+ posts
How unrealistic is a basic LLM made using neural networks in scratch
Yes, but doing so probably isn't allowed because of the chance of inappropriate content from most training data sets, leading to inappropriate answers from the model.cant we use clouds vars to connect scratch project to a external service or smth & run the neural network there i mean there are things like scratch attach Snip
Last edited by BigNate469 (Feb. 6, 2025 02:01:09)
- StarWalker600
-
100+ posts
How unrealistic is a basic LLM made using neural networks in scratch
Sad so its possible but not possible togetherYes, but doing so probably isn't allowed because of the chance of inappropriate content from most training data sets, leading to inappropriate answers from the model.cant we use clouds vars to connect scratch project to a external service or smth & run the neural network there i mean there are things like scratch attach Snip
- 23mConnor
-
25 posts
How unrealistic is a basic LLM made using neural networks in scratch
To some extent it can be possible, like an Ai learning to jump over a lava put for example, deep neural learning in scratch isn’t really possible, it’s like putting the cart before the horse lol.
- CRAZYTVSTU
-
100+ posts
How unrealistic is a basic LLM made using neural networks in scratch
- StarWalker600
-
100+ posts
How unrealistic is a basic LLM made using neural networks in scratch
Looks promising, but it would be slow, not to mention scratch is already slow. Could be possible with this
https://scratch-mit-edu.ezproxyberklee.flo.org/discuss/topic/805201/
But… it can work
We can save pixel data in base 16 million ig cuz there are 16 million possible colors (even more if u take transparency into account). We save it to costume as u said, then we read it using the color detector thing griffpatch made then we convert the base 16 million into smth more sensible like hexadecimal or decimal. We can then use it to essentially write assembly program
But this relies on the fact that there are no asset limit
What ill do is ill make a project to test this. Ill obvs not go for base 16 million cuz thats kinda crazy uk
- CRAZYTVSTU
-
100+ posts
How unrealistic is a basic LLM made using neural networks in scratch
Yeah exactly. Also there's gotta be a point to not optimize the save file. Like, the less optimized the save file is, the less work goes into encoding/decoding it and thus a smaller .json and a larger .png or whatever file type Scratch defaults to. In other words, we would basically have to make each block, each operator, each boolean etc, its own pixel, and how they connect would either be communicated in extra pixels or in pixel orderLooks promising, but it would be slow, not to mention scratch is already slow. Could be possible with this
https://scratch-mit-edu.ezproxyberklee.flo.org/discuss/topic/805201/
But… it can work
We can save pixel data in base 16 million ig cuz there are 16 million possible colors (even more if u take transparency into account). We save it to costume as u said, then we read it using the color detector thing griffpatch made then we convert the base 16 million into smth more sensible like hexadecimal or decimal. We can then use it to essentially write assembly program
But this relies on the fact that there are no asset limit
What ill do is ill make a project to test this. Ill obvs not go for base 16 million cuz thats kinda crazy uk
Edit: Also, theres 480x360 pixels on each costume i BELIEVE (haven't double checked that) which means that each costume can contain 172,800 thingies. Thingies, like, bytes, or whatever the equivalent is here. I suggest we call them Pixelbytes
Edit 2: I'm working on a project atm and the file is above 100MB and it's saving properly so I'm fairly certain there isn't a project size limit
Last edited by CRAZYTVSTU (Feb. 7, 2025 19:14:03)
- StarWalker600
-
100+ posts
How unrealistic is a basic LLM made using neural networks in scratch
what im doing is making a virtual machine (tada the SVM scratch virtual machine). So we will hv to program in assembly cuz thats more simpler to compile into scratch blocksYeah exactly. Also there's gotta be a point to not optimize the save file. Like, the less optimized the save file is, the less work goes into encoding/decoding it and thus a smaller .json and a larger .png or whatever file type Scratch defaults to. In other words, we would basically have to make each block, each operator, each boolean etc, its own pixel, and how they connect would either be communicated in extra pixels or in pixel orderLooks promising, but it would be slow, not to mention scratch is already slow. Could be possible with this
https://scratch-mit-edu.ezproxyberklee.flo.org/discuss/topic/805201/
But… it can work
We can save pixel data in base 16 million ig cuz there are 16 million possible colors (even more if u take transparency into account). We save it to costume as u said, then we read it using the color detector thing griffpatch made then we convert the base 16 million into smth more sensible like hexadecimal or decimal. We can then use it to essentially write assembly program
But this relies on the fact that there are no asset limit
What ill do is ill make a project to test this. Ill obvs not go for base 16 million cuz thats kinda crazy uk
Edit: Also, theres 480x360 pixels on each costume i BELIEVE (haven't double checked that) which means that each costume can contain 172,800 thingies. Thingies, like, bytes, or whatever the equivalent is here. I suggest we call them Pixelbytes
EDIT1: We can then use the color values to determine which assembly instruction is written, along with its parameters. a different project can be used to make programs
EDIT2: Lets continue this conversation on ur topic as this is going off topic
EDIT3: Is this approach good or should I do some other way???
Last edited by StarWalker600 (Feb. 7, 2025 19:07:24)
- StarWalker600
-
100+ posts
How unrealistic is a basic LLM made using neural networks in scratch
BUMP
- Discussion Forums
- » Project Ideas
-
» How unrealistic is a basic LLM made using neural networks in scratch