Discuss Scratch

StarWalker600
Scratcher
100+ posts

How unrealistic is a basic LLM made using neural networks in scratch

Title.

EDIT1: For those wondering whats an llm its large lanuage model eg. chatgpt, gemini, deepseek, claude, etc
EDIT2: Pls give me positive coments

Last edited by StarWalker600 (Feb. 2, 2025 14:34:13)

BigNate469
Scratcher
1000+ posts

How unrealistic is a basic LLM made using neural networks in scratch

Basically impossible- the training data alone would exceed the project.json maximum size limit.

Not to mention that without GPU acceleration it would be horribly slow.
StarWalker600
Scratcher
100+ posts

How unrealistic is a basic LLM made using neural networks in scratch

BigNate469 wrote:

Basically impossible- the training data alone would exceed the project.json maximum size limit.

Not to mention that without GPU acceleration it would be horribly slow.
cant we use clouds vars to connect scratch project to a external service or smth & run the neural network there i mean there are things like scratch attach
BigNate469
Scratcher
1000+ posts

How unrealistic is a basic LLM made using neural networks in scratch

StarWalker600 wrote:

BigNate469 wrote:

Snip
cant we use clouds vars to connect scratch project to a external service or smth & run the neural network there i mean there are things like scratch attach
Yes, but doing so probably isn't allowed because of the chance of inappropriate content from most training data sets, leading to inappropriate answers from the model.

Last edited by BigNate469 (Feb. 6, 2025 02:01:09)

StarWalker600
Scratcher
100+ posts

How unrealistic is a basic LLM made using neural networks in scratch

BigNate469 wrote:

StarWalker600 wrote:

BigNate469 wrote:

Snip
cant we use clouds vars to connect scratch project to a external service or smth & run the neural network there i mean there are things like scratch attach
Yes, but doing so probably isn't allowed because of the chance of inappropriate content from most training data sets, leading to inappropriate answers from the model.
Sad so its possible but not possible together
23mConnor
Scratcher
25 posts

How unrealistic is a basic LLM made using neural networks in scratch

To some extent it can be possible, like an Ai learning to jump over a lava put for example, deep neural learning in scratch isn’t really possible, it’s like putting the cart before the horse lol.
CRAZYTVSTU
Scratcher
100+ posts

How unrealistic is a basic LLM made using neural networks in scratch

StarWalker600
Scratcher
100+ posts

How unrealistic is a basic LLM made using neural networks in scratch

CRAZYTVSTU wrote:

Could be possible with this

https://scratch-mit-edu.ezproxyberklee.flo.org/discuss/topic/805201/
Looks promising, but it would be slow, not to mention scratch is already slow.
But… it can work
We can save pixel data in base 16 million ig cuz there are 16 million possible colors (even more if u take transparency into account). We save it to costume as u said, then we read it using the color detector thing griffpatch made then we convert the base 16 million into smth more sensible like hexadecimal or decimal. We can then use it to essentially write assembly program

But this relies on the fact that there are no asset limit

What ill do is ill make a project to test this. Ill obvs not go for base 16 million cuz thats kinda crazy uk
CRAZYTVSTU
Scratcher
100+ posts

How unrealistic is a basic LLM made using neural networks in scratch

StarWalker600 wrote:

CRAZYTVSTU wrote:

Could be possible with this

https://scratch-mit-edu.ezproxyberklee.flo.org/discuss/topic/805201/
Looks promising, but it would be slow, not to mention scratch is already slow.
But… it can work
We can save pixel data in base 16 million ig cuz there are 16 million possible colors (even more if u take transparency into account). We save it to costume as u said, then we read it using the color detector thing griffpatch made then we convert the base 16 million into smth more sensible like hexadecimal or decimal. We can then use it to essentially write assembly program

But this relies on the fact that there are no asset limit

What ill do is ill make a project to test this. Ill obvs not go for base 16 million cuz thats kinda crazy uk
Yeah exactly. Also there's gotta be a point to not optimize the save file. Like, the less optimized the save file is, the less work goes into encoding/decoding it and thus a smaller .json and a larger .png or whatever file type Scratch defaults to. In other words, we would basically have to make each block, each operator, each boolean etc, its own pixel, and how they connect would either be communicated in extra pixels or in pixel order

Edit: Also, theres 480x360 pixels on each costume i BELIEVE (haven't double checked that) which means that each costume can contain 172,800 thingies. Thingies, like, bytes, or whatever the equivalent is here. I suggest we call them Pixelbytes

Edit 2: I'm working on a project atm and the file is above 100MB and it's saving properly so I'm fairly certain there isn't a project size limit

Last edited by CRAZYTVSTU (Feb. 7, 2025 19:14:03)

StarWalker600
Scratcher
100+ posts

How unrealistic is a basic LLM made using neural networks in scratch

CRAZYTVSTU wrote:

StarWalker600 wrote:

CRAZYTVSTU wrote:

Could be possible with this

https://scratch-mit-edu.ezproxyberklee.flo.org/discuss/topic/805201/
Looks promising, but it would be slow, not to mention scratch is already slow.
But… it can work
We can save pixel data in base 16 million ig cuz there are 16 million possible colors (even more if u take transparency into account). We save it to costume as u said, then we read it using the color detector thing griffpatch made then we convert the base 16 million into smth more sensible like hexadecimal or decimal. We can then use it to essentially write assembly program

But this relies on the fact that there are no asset limit

What ill do is ill make a project to test this. Ill obvs not go for base 16 million cuz thats kinda crazy uk
Yeah exactly. Also there's gotta be a point to not optimize the save file. Like, the less optimized the save file is, the less work goes into encoding/decoding it and thus a smaller .json and a larger .png or whatever file type Scratch defaults to. In other words, we would basically have to make each block, each operator, each boolean etc, its own pixel, and how they connect would either be communicated in extra pixels or in pixel order

Edit: Also, theres 480x360 pixels on each costume i BELIEVE (haven't double checked that) which means that each costume can contain 172,800 thingies. Thingies, like, bytes, or whatever the equivalent is here. I suggest we call them Pixelbytes
what im doing is making a virtual machine (tada the SVM scratch virtual machine). So we will hv to program in assembly cuz thats more simpler to compile into scratch blocks

EDIT1: We can then use the color values to determine which assembly instruction is written, along with its parameters. a different project can be used to make programs
EDIT2: Lets continue this conversation on ur topic as this is going off topic
EDIT3: Is this approach good or should I do some other way???

Last edited by StarWalker600 (Feb. 7, 2025 19:07:24)

StarWalker600
Scratcher
100+ posts

How unrealistic is a basic LLM made using neural networks in scratch

BUMP

Powered by DjangoBB