Posts on X before May 10, 2024, 2:29:38 PM UTC

anyone who tells you to "just put it all in the context window, it will be better" doesn't know how context length works for LLMs nor have they tested it out extensively enough to notice the subtle retardation that happens (well, first subtle, then it's very obvious as you continue to add context) don't get me wrong; there's a sweet spot. obviously if you put too little context, the result will be worse than if you put just the right amount of context; but past that, the more you put in, aka the more garbage you put in after what's immediately necessary, the worse your end results will end up being

agreed. apple is primarily known for its security and privacy. scoff all you want, android users/apple haters; there's no better mainstream device that prioritizes security other than apple products. no, your fringe modular rooted android with sys component replacements doesn't count.

the problem I have with these takes is that... what other choice do you have? you need to use AI in your business? then why wouldn't you use the SOTA choice? which will only get better? take perplexity for example. yeah, they're predictably going to face competition with openai's new search that they're releasing soon. but... so what? yeah, I guess they could have used... their own self hosted models? gone a different business route altogether? not done perplexity in the first place? pivoted earlier? one version is you make a lot of money with the possibility of being replicated by the service you are benefiting from. the other version is... you don't make money? you choose something else that... doesn't have to do with AI?

OK, I've seen this type of behavior A LOT recently from ALL the llama 3 models. here's what i think is going on: first, there was no need for it to make up a word. it could have just used... preface. "I must preface" would have worked just fine from heavy personal experience with all the models, it seems like they always try to show off their "humanity" within the first part and last part of their responses. it often starts off with something cringey along the lines of "Brilliant! What an interesting take." - and then it gives you the actual answer to your query / conversation similarly, it ends the same way as well - many times, I've had it end with a response like "so there you have it! Thanks for listening (or, READING - heh) and I hope I answered your question there." yes, this satisfies those that begged for a more humane talking companion but it also seems to introduce these creative(?) elements as seen here: making up words, catching itself as a human who made a speech mistake would, etc. I think it's a good step towards a less robotic, more enjoyable conversational system, with 1 caveat; I want to be able to turn this type of behavior off. hopefully one can do so through some additional system prompt rulesets (though personally I haven't reached a point where robotic behavior is favorable, so I haven't tried)