Yeah, I’m in the same boat.

There’s also PeerTube, the Fediverse counterpart to YouTube. Unfortunately, while there’s some good stuff you can find (and some re-uploads of YouTube), there’s just not as much content. I’d imagine the userbase is pretty small, too.

I use a cheap VPS to host my email server. It’s a bit easier than running it solely at home, but there’s a lot of annoying work to “verify” yourself. Once you get your DNS records good, you shouldn’t be blocked after that (unlike a home server). It only costs me $5/month plus the domain, which I think is money well spent. Doing the admin work to make sure I’m secure still needs to happen, but I don’t mind that work and find it fun.

The original paper itself, for those who are interested.

Overall, this is really interesting research and a really good “first step.” I will be interested to see if this can be replicated on other models. One thing that really stood out, though, was that certain details are obfuscated because of Sonnet being proprietary. Hopefully follow-on work is done on one of the open source models to confirm the method.

One of the notable limitations is quantifying activation’s correlation to text meaning, which will make any sort of controls difficult. Sure, you can just massively increase or decrease a weight, and for some things that will be fine, but for real manual fine tuning, that will prove to be a difficulty.

I suspect this method is likely generalizable (maybe with some tweaks?), and I’d really be interested to see how this type of analysis could be done on other neural networks.

You’re getting downvoted, but you’re right. And that is the reason that using proprietary software and SaaS is a problem. If I’m only buying the right to use a copy of something as a company sees fit, then I’m not really buying anything. I’m essentially paying a company a tribute to use their software in their way.

Decades ago, it was the same way, but it felt different. We got physical media, and we could do what we wished with the files: modify them, delete them, etc. Hell, the EULAs for some '90s and early '00s software even said you could use the software in perpetuity, and we could use software in anyway we saw fit. The biggest constraint was on selling copies. Back then, and even now, that seems pretty reasonable. (Though, as an aside, it would have been better to also get access to the source code, but I digress.)

Now, we have to use company’s software exactly how they want us to use it. Personally, I refuse to go along with this (as much as I can), so I have migrated most of my digital life to FLOSS.

No lie, that actually sounds kinda good and I want to try it.

Not necessarily. The Free and Libre Open Source Software (FLOSS) movement is a thing. Most of the Fediverse is FLOSS, and I doubt there’s anyone who can take Lemmy or Mastodon closed source and buy every instance and then stop pop-up instances. It does require quite a bit of work, though, so it is difficult.

I think the real challenging thing is that a great FLOSS service needs to attract attention and care. When I bring up Fediverse/FLOSS alternatives to software my friends complain about, I’m met with lukewarm-at-best reactions, generally due to networking effects (I think).

I think a big difference, though, is that there is political force to ending TikTok. The US government has no major issues with Twitter, Facebook, Reddit, etc. existing. Remember, there’s actual legislation banning TikTok. Whether that makes a real difference or not, well, I guess we just wait and see. Personally, I think they all should go down in flames.

It’s not just convenient for them to do it; it’s how they are able to evade anti-trust action (not that the U.S. is great at it anyway but still). I also run my own mail server. It’s not impossible, and I wouldn’t even say it’s even hard. It’s just time consuming to set up (if it’s the first time), and there are a lot of hurdles to make it so impractical that it’s virtually impossible to the average person. Only the most patient or those who have a real desire to run their own mail server will even attempt it. Anyone can set up their own mail server, but most won’t because it’s not worth it compared to using something that just works from Google.

I work/study in AI, and it is completely over-hyped. For one thing, the C-suite can’t wrap it’s head around the fact that AI != LLM; they all seem to think all AI is just LLMs. On top of that, they are way too eager to throw humans out of the loop.

That said, I think LLM applications, even in their current form, are super useful in development and business practices. I myself use it to increase my productivity in coding. But, I use it as an augmentation rather than a replacement. One of my friends put it best the other day, “LLMs are like a junior dev to your senior dev. You need to be hyper-specific, and you need to check it’s output.” In other words, it’s great for off-loading some work, but it isn’t going to completely replace humans.

With that said, I’m a bit annoyed that other AI fields are being over-shadowed by LLMs. There’s a ton of other interesting work being done in those fields that is super useful and important. All of them, though, are not going to replace humans but rather augment and make humans more productive. I’ve found that an AI-Human team is most effective.

You’re not wrong. I remember how Bush, McCain, Romney, Obama, Clinton, and others were called Nazis at different points. While it was never really taken seriously then (as it shouldn’t have been), the term has become virtually meaningless. Where the term was reserved for the worst-of-the-worst, for years, it was invoked at the slightest disagreement. Now that there’s a literal Nazi-adjacent person running and getting called out for it, it falls flat.

Except that scaling alone won’t lead to AGI. It may generate better, more convincing text, but the core algorithm is the same. That “special juice” is almost certainly going to come from algorithmic development rather than just throwing more compute at the problem.

I mean, that’s more-or-less what I said. We don’t know the theoretical limits of how good that text generation is when throwing more compute at it and adding parameters for the context window. Can it generate a whole book that is fairly convincing, write legal briefs off of the sum of human legal knowledge, etc.? Ultimately, the algorithm is the same, so like you said, the same problems persist, and the definition of “better” is wishy-washy.

Cool, Bill Gates has opinions. I think he’s being hasty and speaking out of turn and only partially correct. From my understanding, the “big innovation” of GPT-4 was adding more parameters and scaling up compute. The core algorithms are generally agreed to be mostly the same from earlier versions (not that we know for sure since OpenAI has only released a technical report). Based on that, the real limit on this technology is compute and number of parameters (as boring as that is), and so he’s right that the algorithm design may have plateaued. However, we really don’t know what will happen if truly monster rigs with tens-of-trillions of parameters are used when trained on the entirety of human written knowledge (morality of that notwithstanding), and that’s where he’s wrong.

It depends on the terms of employment. If they are salaried, then there are no real work hours and just work to do. In general, if someone is salaried, they’re paid to do a job not when they do it.

This is a much better article. OP’s article just shows the author’s surface understanding of how coding works and how well an LLM can actually code. There’s way more that goes into a programming task than just coding.

I see LLMs as having the potential of being almost like a super library. I can prompt GPT, Claude, etc. to write me a custom function that I copy, paste, test, scrutinize, and almost certainly change. It’s a tool that will make someone a more productive programmer. It won’t completely subsume a human’s ability to be creative and put the pieces together.

At the absolute worst over the next decade, I could see programming changing from writing and debugging code to prompting, stitching together, and debugging.

It works well if you know how to prompt well. LLMs are able to do so much, but a user just needs to know how to use it correctly. The technology is still in its infancy, so it’s a bit difficult to use well.

I think it’s less that “media literacy is not taught” but that media literacy is not learned. Like @audiomodder said, everyone is graduated regardless. So, on one hand, there are students who either will not or cannot learn the material (for one reason or another, such as disability, stress, family, etc.) and teachers who get a laundry list of things to teach and not enough time or support to teach it.

Ultimately, the problem is a lack of focus on education as a society. Children are pulled in too many directions, and teachers aren’t given the resources needed, so we end up with a broken educational system.

There’s a bit of truth to that. Honestly, leadership via email/chat is hard. Bad managers can’t micromanage and pretend to know what’s going on, so they get screwed. Good managers (which I promise do exist) have a hard time really getting their team to excel and know what the real problems are that they can address and fix. So, every team ends up in a range of mediocrity either rising from not having to deal with useless managers or not able to reach their full potential with good managers.

Is that bad? Eh, probably not.

This is so true. Although not popular with people in my field (Computer Science), I highly value my liberal arts and humanities education. Far from being “worthless mandatory classes” (like many in my major believed), I found thee classes to be the most enlightening. I learned to wield algorithms and build software in my major, but I learned why I should and how to think critically about how to do it to best serve people (not to mention just simple communication). If anything, there should be more liberal arts in the curriculum, not less.