THIS SUGGESTION ISN'T TRYING TO PREVENT PROJECTS WITH BAD WORDS FROM BEING SHARED, IT'S TRYING TO NOTIFY THE SCRATCH TEAM ABOUT THESE PROJECTS AS SOON AS POSSIBLE.
IT ALSO DOES NOT CENSOR ANYTHING.

Troll projects that contain curse words and bad words are always annoying. They have to be manually reported, which means that some poor scratcher has to see them before it gets taken down. But what if we could fix that? What if, we could take down an inappropriate project the second it's shared?

My suggestion today is that when a project is shared, a filter goes through the project. However, if it detects something inappropriate, it won't censor it. Rather, it will send a report to the report queue. This way, bad projects get the scratch team's attention as soon as possible, and no one has to view the inappropriate things and projects that the filter is misdetecting don't experience any issues like they normally do when the filter makes a mistake.

also, trolls could in theory make multiple accounts and share many inappropriate projects. one way to solve this would be to have a system in place where if the project filter triggers twice in less than one minute, the user is blocked until the projects are reviewed by the scratch team. I seriously doubt that anyone who's accidentally triggering the filter will manage to do so twice in one minute.

Pros:

• gets the scratch team's attention to inappropriate projects as soon as possible

• bad projects can be taken down quickly, which gets fewer people(if not none at all) to view them.
  

• because the filter doesn't censor anything, people won't experience issues like they normally do when the filter makes a mistake
  

• this will work because trolls often use the normal text inputs, and because they wouldn't know this exists, they probably wouldn't use anything else.
  

Cons:

• since many projects are shared each day, the report queue might fill up. However, I doubt this will happen because the filter only sends a report if it detects something bad, meaning only projects with bad words and projects that were misdetected by the filter will go to the report queue. Also, the point of the report queue is to have a list of inappropriate projects so that they can be taken down. The more, the merrier.
  

• There are workarounds to this filter. Luckily this filter does everything it can to make its existence known to absolutely no one, thus motivating no one to use the workarounds.

Project filtering is certainly an area that we are very interested in.

Great idea! What kind of things would this filter check specifically?

musicROCKS013 wrote:

Great idea! What kind of things would this filter check specifically?

things normally filter checked by the comment filter, e.g. curse words, personal information, etc.

-Valtren- wrote:

things normally filter checked by the comment filter, e.g. curse words, personal information, etc.

Ok, but- Like, where would it check? In all text inputs in the project?

musicROCKS013 wrote:

-Valtren- wrote:

things normally filter checked by the comment filter, e.g. curse words, personal information, etc.

Ok, but- Like, where would it check? In all text inputs in the project?

yeah, I guess. an AI to check through costumes would also be nice, but would be expensive and require a lot of processing power.
most trolls use the normal text inputs, so we know for sure that the bad words will be detected(and even better, they won't try to work around it because they don't even know that it's being detected).

-Valtren- wrote:

yeah, I guess. an AI to check through costumes would also be nice, but would be expensive and require a lot of processing power.

How about if there was a thing that checked the costumes it would only check the text inputs in the vector mode. Also for costume names, variable names, and inputs for blocks like this

(join [] [world])

musicROCKS013 wrote:

-Valtren- wrote:

yeah, I guess. an AI to check through costumes would also be nice, but would be expensive and require a lot of processing power.

How about if there was a thing that checked the costumes it would only check the text inputs in the vector mode. Also for costume names, variable names, and inputs for blocks like this

(join [] [world])

yeah, that's what I meant

MEGA SUPPORT

Sounds like a good suggestion, but it sounds like to me the project would still be shared even though there's inappropriate content on it? I think a better idea would be as soon as the filter detects something inappropriate, a pop-up would show up saying something like ‘Hey, our filter noticed you wrote something that’s against Scratch's guidelines, (shows what needs to be removed), please remove this before sharing, thank you!'. This would prevent the project from being even shared in the first place.

EeveeLegends wrote:

Sounds like a good suggestion, but it sounds like to me the project would still be shared even though there's inappropriate content on it? I think a better idea would be as soon as the filter detects something inappropriate, a pop-up would show up saying something like ‘Hey, our filter noticed you wrote something that’s against Scratch's guidelines, (shows what needs to be removed), please remove this before sharing, thank you!'. This would prevent the project from being even shared in the first place.

true, but it would alert the scratch team as soon as it's shared. this means that it'll probably get taken down before anyone has any time to even view it.
also, the filter can make mistakes sometimes. imagine how annoying it would be if you couldn't share a project just because the filter misdetected something. this way, innocent projects can still be shared, while inappropriate projects can get taken down more easily

EeveeLegends wrote:

Sounds like a good suggestion, but it sounds like to me the project would still be shared even though there's inappropriate content on it? I think a better idea would be as soon as the filter detects something inappropriate, a pop-up would show up

Honestly this aspect is one of the good ones of the suggestion, because there are now little to no false positives keeping your project from getting shared. Like, you know how annoying users find the comment filter? How much worse would this be if you couldn't share projects! I think -Valtren-'s original idea is a great solution to this problem.

-Valtren- wrote:

EeveeLegends wrote:

Sounds like a good suggestion, but it sounds like to me the project would still be shared even though there's inappropriate content on it? I think a better idea would be as soon as the filter detects something inappropriate, a pop-up would show up saying something like ‘Hey, our filter noticed you wrote something that’s against Scratch's guidelines, (shows what needs to be removed), please remove this before sharing, thank you!'. This would prevent the project from being even shared in the first place.

true, but it would alert the scratch team as soon as it's shared. this means that it'll probably get taken down before anyone has any time to even view it.
also, the filter can make mistakes sometimes. imagine how annoying it would be if you couldn't share a project just because the filter misdetected something. this way, innocent projects can still be shared, while inappropriate projects can get taken down more easily

I thought about the filter detecting something wrong, and I think it's not that much of a nuisance. If the filter accidentally deems a comment inappropriate, the person could make the comment later, same goes with projects. Also.. the ST might not take down the project super quickly and someone still the chance to see it. It's just beneficial to prevent the problem from even happening.

EeveeLegends wrote:

I thought about the filter detecting something wrong, and I think it's not that much of a nuisance.

Most people do.

EeveeLegends wrote:

If the filter accidentally deems a comment inappropriate, the person could make the comment later, same goes with projects.

But what if it's a false positive? And it happens multiple times throughout the project? Okay, so thinking about it, I do like your idea, but I think that it should only be reserved for outright swears. Like, the f-word obviously would cause the project to go “woah woah woah” but the phrase “tweet” or “discord” would only send an alert..?

EeveeLegends wrote:

Also.. the ST might not take down the project super quickly and someone still the chance to see it.

How much greater would the chance be if there WAS no filter? If even one person was spared from seeing a project from this, this feature will have helped.

EeveeLegends wrote:

It's just beneficial to prevent the problem from even happening.

I agree with you, but most of the time the filter is just stupid and can't detect context very well. For example, “bit challenging,” and “go down on,” are both blocked by the filter, despite being able to be used in very innocent contexts. For example, “Alright, this next part gets a bit challenging!” or “Once you go down on the ladder, find the key.” or simple things that could stop a project from getting shared.

musicROCKS013 wrote:

EeveeLegends wrote:

If the filter accidentally deems a comment inappropriate, the person could make the comment later, same goes with projects.

But what if it's a false positive? And it happens multiple times throughout the project? Okay, so thinking about it, I do like your idea, but I think that it should only be reserved for outright swears. Like, the f-word obviously would cause the project to go “woah woah woah” but the phrase “tweet” or “discord” would only send an alert..?

I think this is a pretty good middle ground to mine and Valtren's suggestion. But I don't know if the filter can prioritize certain words or if it treats all flaggable words the same. If it can prioritize words, then this would be a really good idea to implement.

EeveeLegends wrote:

But I don't know if the filter can prioritize certain words or if it treats all flaggable words the same.

I bet there could be a “grey list” of words that are iffy, but not completely unallowed. I bet this is possible, it's just the question of what words to unclude.

musicROCKS013 wrote:

EeveeLegends wrote:

But I don't know if the filter can prioritize certain words or if it treats all flaggable words the same.

I bet there could be a “grey list” of words that are iffy, but not completely unallowed. I bet this is possible, it's just the question of what words to unclude.

I think the words that would fit best on this “grey list” would be words that have multiple contexts with one of them being iffy or against ST guidelines. For example, Discord, it could mean the actual word, the MLP character, or the website. The ST could come up with a good grey list using this as a base guideline, they probably have to check this kind of stuff all the time.

EeveeLegends wrote:

Sounds like a good suggestion, but it sounds like to me the project would still be shared even though there's inappropriate content on it? I think a better idea would be as soon as the filter detects something inappropriate, a pop-up would show up saying something like ‘Hey, our filter noticed you wrote something that’s against Scratch's guidelines, (shows what needs to be removed), please remove this before sharing, thank you!'. This would prevent the project from being even shared in the first place.

It must NOT show what's wrong to avoid learning inappropriate content in the case of a lack of knowledge.

With the current system, if a small user shares an inappropriate project, it takes longer to get noticed and consequently to be reported, meaning small users have a bit of leeway.

EeveeLegends wrote:

musicROCKS013 wrote:

EeveeLegends wrote:

But I don't know if the filter can prioritize certain words or if it treats all flaggable words the same.

I bet there could be a “grey list” of words that are iffy, but not completely unallowed. I bet this is possible, it's just the question of what words to unclude.

I think the words that would fit best on this “grey list” would be words that have multiple contexts with one of them being iffy or against ST guidelines. For example, Discord, it could mean the actual word, the MLP character, or the website. The ST could come up with a good grey list using this as a base guideline, they probably have to check this kind of stuff all the time.

at this point I would recommend making your own suggestion