"Tweet id","Tweet permalink","Tweet text","time","impressions","engagements","engagement rate","retweets","replies","likes","user profile clicks","url clicks","hashtag clicks","detail expands","permalink clicks","app opens","app installs","follows","email tweet","dial phone","media views","media engagements","promoted impressions","promoted engagements","promoted engagement rate","promoted retweets","promoted replies","promoted likes","promoted user profile clicks","promoted url clicks","promoted hashtag clicks","promoted detail expands","promoted permalink clicks","promoted app opens","promoted app installs","promoted follows","promoted email tweet","promoted dial phone","promoted media views","promoted media engagements"
"1790808530989056208","https://twitter.com/gwern/status/1790808530989056208","@mayfer (If they're doing it the obvious way by just sequence-prediction on tokens interleaved, I would expect so, although it might be different enough from the overwhelming bulk of the corpus to require some few-shot meta-learning of the intended interleaving pattern.)","2024-05-15 18:16 +0000","331.0","15.0","0.045317220543806644","0.0","0.0","4.0","3.0","0.0","0.0","8.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790807220340289736","https://twitter.com/gwern/status/1790807220340289736","@Dr_CMingarelli You are reinventing ""flattened logits"" which was observed back in like 2022, is caused by RLHF training, and was also documented in the GPT-4 paper: https://t.co/RVxreOCQBE","2024-05-15 18:11 +0000","279.0","22.0","0.07885304659498207","0.0","0.0","3.0","0.0","10.0","0.0","9.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790775368737030274","https://twitter.com/gwern/status/1790775368737030274","@kindgracekind This reminds me of trying to talk with hedge fund guys about anything interesting. ""So what are you up to nowadays? Doing anything cool with DL?"" ""Yes, it involves... stocks. Wait, I've said too much, forget I said anything.""","2024-05-15 16:04 +0000","540.0","27.0","0.05","0.0","2.0","13.0","0.0","0.0","0.0","12.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790771914132111504","https://twitter.com/gwern/status/1790771914132111504","@TheZvi Yes. Much more common than with GPT-4, where I hardly ever remember it. And it just keeps doing it too even if you ask explicitly about it or tell it not to.","2024-05-15 15:51 +0000","752.0","32.0","0.0425531914893617","0.0","1.0","8.0","3.0","0.0","0.0","20.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790567220512231492","https://twitter.com/gwern/status/1790567220512231492","@jordanschnyc https://t.co/NwE1IfiUFM","2024-05-15 02:17 +0000","111.0","36.0","0.32432432432432434","0.0","0.0","5.0","0.0","0.0","0.0","22.0","0.0","0","0","0","0","0","9","9","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790565930902094274","https://twitter.com/gwern/status/1790565930902094274","@friedmandave @TheRealRyanRose That seems like the sort of review which would turn up the fewest errors of the sort a LLM would make.","2024-05-15 02:12 +0000","142.0","24.0","0.16901408450704225","0.0","1.0","1.0","9.0","0.0","0.0","13.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790499334271668504","https://twitter.com/gwern/status/1790499334271668504","@shaggysurvives Ask him what ""minutiae"", exactly, we are connecting with in a clockwork robot man winding down due to lack of pneumatic pressure.","2024-05-14 21:48 +0000","1290.0","54.0","0.04186046511627907","0.0","1.0","31.0","12.0","0.0","0.0","10.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790480547837256164","https://twitter.com/gwern/status/1790480547837256164","@Ted_Underwood ChatGPT-4o: not compelling at all, but after many iterations, still hadn't degenerated or begun rhyming, so that's good. https://t.co/3fdLhIrttC","2024-05-14 20:33 +0000","335.0","42.0","0.1253731343283582","0.0","0.0","3.0","1.0","0.0","0.0","14.0","0.0","0","0","0","0","0","24","24","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790453724550152440","https://twitter.com/gwern/status/1790453724550152440","@tszzl ""His followers called him Mahasamaltman, or the Vajrapreneur, the Turingsattva who taught the Silicon Samadhi. He preferred 'Samathman', or just, Sam.
He never claimed to be a dev(a); but then, he never claimed to not be a dev either.
Circumstances being what they were, neither¡","2024-05-14 18:46 +0000","1939.0","174.0","0.08973697782362042","0.0","1.0","39.0","9.0","6.0","0.0","119.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790444114560852371","https://twitter.com/gwern/status/1790444114560852371","@0xrin1 @nearcyan @elonmusk This isn't crypto twitter, this is AI Twitter!","2024-05-14 18:08 +0000","54.0","7.0","0.12962962962962962","0.0","1.0","1.0","0.0","0.0","0.0","5.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790432792934760559","https://twitter.com/gwern/status/1790432792934760559","@0xrin1 @nearcyan @elonmusk What's a 'CT'?","2024-05-14 17:23 +0000","58.0","6.0","0.10344827586206896","0.0","1.0","0.0","0.0","0.0","0.0","5.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790432599376027697","https://twitter.com/gwern/status/1790432599376027697","@zacharynado IIRC, the consensus at the time was that it was a large VAE trained separately, then frozen and used with cross-attention in a GPT-4. Which was why it was so expensive and not as good as one would expect from a truly multi-modal 'tokenize all the things!' scaleup.","2024-05-14 17:22 +0000","439.0","23.0","0.05239179954441914","0.0","0.0","4.0","2.0","0.0","0.0","17.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790430750262575557","https://twitter.com/gwern/status/1790430750262575557","@nearcyan @elonmusk (Could you go further? ofc. Like my curated similars, it would be useful for tweet authors to approve a list of recommended-tweets. Dumping a list of Twitter URLs into an appended tweet is doable but largely useless: a lot of work, no previews/transclusion for all but 1 etc)","2024-05-14 17:15 +0000","637.0","19.0","0.029827315541601257","0.0","1.0","2.0","2.0","0.0","0.0","14.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790429912676831660","https://twitter.com/gwern/status/1790429912676831660","@nearcyan @elonmusk (Once I had that working, it could be integrated with writing draft tweets: embed drafts in the background, and retrieve relevant tweets live also in another pane.)","2024-05-14 17:12 +0000","787.0","19.0","0.0241423125794155","0.0","1.0","1.0","2.0","1.0","0.0","14.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790429536594583890","https://twitter.com/gwern/status/1790429536594583890","@nearcyan @elonmusk For Twitter, I think the way I'd design it would be much more aggressive about quality-control & consolidating into blocks; maybe include an N count icon; and then by default, transclude the best ones into the right margin if enough horizontal space, or else append them.","2024-05-14 17:10 +0000","488.0","17.0","0.03483606557377049","0.0","1.0","2.0","2.0","0.0","0.0","12.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790428872535572881","https://twitter.com/gwern/status/1790428872535572881","@nearcyan @elonmusk The bigger issue is that you don't know if 'related' are any good, so you get trained out of never using it by effort+risk...
This is why I have friction-less on-hover popups for https://t.co/assIVk4RfF 'similar links', set min distance, & curate subsets for showing by default.","2024-05-14 17:08 +0000","525.0","40.0","0.0761904761904762","0.0","1.0","13.0","3.0","4.0","0.0","19.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790413534460780939","https://twitter.com/gwern/status/1790413534460780939","@suchenzang @xiaoqianWX Eh. The mere fact that these are there points to how sloppy the tokenization process is. Maybe they didn't bother to QA the Chinese tokens at all or make sure they had a Chinese NSFW filter. Wouldn't be the first NSFW-related filter screwup, IMO (like the DALL-E 2 anime?).","2024-05-14 16:07 +0000","303.0","21.0","0.06930693069306931","0.0","1.0","2.0","1.0","0.0","0.0","17.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790413050543042710","https://twitter.com/gwern/status/1790413050543042710","@cHHillee @cosmojg @teortaxesTex Oh, that's a good point. Yeah, a boatload of H200s does sound more plausible than B100s right now.","2024-05-14 16:05 +0000","157.0","27.0","0.17197452229299362","0.0","0.0","4.0","1.0","0.0","0.0","21.0","0.0","0","0","0","0","0","1","1","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790411950112817513","https://twitter.com/gwern/status/1790411950112817513","@cHHillee @cosmojg @teortaxesTex I didn't watch the Murati video, but wasn't there a mention of them having novel hardware for this and thanking Jensen in particular? That sounds like a B100 ref.","2024-05-14 16:00 +0000","259.0","30.0","0.11583011583011583","0.0","2.0","7.0","1.0","0.0","0.0","20.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790409553093312822","https://twitter.com/gwern/status/1790409553093312822","@xiaoqianWX @suchenzang Yeah, isn't being confused or conflated a common behavior of glitch or undertrained tokens? Since OA dumped in a huge new batch of bad tokens, there's now more than ever.","2024-05-14 15:51 +0000","150.0","6.0","0.04","0.0","1.0","0.0","1.0","0.0","0.0","4.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790389220969685137","https://twitter.com/gwern/status/1790389220969685137","@joodalooped @kaihenthoiwane Yes, it looks pretty cool! I'm not sure how much time I'll spend generating new fonts, though. No one seemed to care about the dropcats - it didn't get reshared anywhere...","2024-05-14 14:30 +0000","53.0","3.0","0.05660377358490566","0.0","0.0","0.0","0.0","0.0","0.0","3.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790388943118106627","https://twitter.com/gwern/status/1790388943118106627","@cosmojg @teortaxesTex So you have something like a 10t-parameter GPT with heavy MoEs specializing per-modality on separate new Nvidia B100s spitting out a handful of VAE-like tokens (maybe 200k, reusing the BPE range with per-modality delimiters) for low latency + high utilization.","2024-05-14 14:29 +0000","1066.0","50.0","0.04690431519699812","0.0","2.0","8.0","10.0","0.0","0.0","28.0","0.0","0","0","0","0","0","2","2","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790388455190475042","https://twitter.com/gwern/status/1790388455190475042","@cosmojg @teortaxesTex If I had to guess, I'm thinking a combination of aggressive sparsity/MoE (MoEs make a huge amount of sense for multi-modality like this because come on, how much transfer/parameter-sharing could there possibly be between 'female voice' and 'Python code'?), VAEs, & new hardware.","2024-05-14 14:27 +0000","1023.0","56.0","0.05474095796676442","0.0","1.0","9.0","8.0","0.0","0.0","35.0","0.0","0","0","0","0","0","3","3","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790205700293214346","https://twitter.com/gwern/status/1790205700293214346","@andy_matuschak (Sequel: ""...someone broke into our apartment and left a second book about implementing quantum computer hardware ?"")","2024-05-14 02:21 +0000","839.0","26.0","0.03098927294398093","0.0","1.0","15.0","4.0","0.0","0.0","6.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790205123429364188","https://twitter.com/gwern/status/1790205123429364188","@tianle_cai This is funny because it would take like <$1 of tokens to have GPT-4 loop through the 100k and for each one answer a prompt like ""Is this a bad phrase unlikely to be used in high-quality writing or code? Translate if necessary"" before you hardwire it into your $100m or $1b model","2024-05-14 02:18 +0000","321.0","59.0","0.1838006230529595","0.0","0.0","32.0","11.0","0.0","0.0","16.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790189403115057508","https://twitter.com/gwern/status/1790189403115057508","@itsandrewgao (BPEisms: ""Groteful"", ""of of"")","2024-05-14 01:16 +0000","751.0","23.0","0.03062583222370173","0.0","1.0","7.0","5.0","0.0","0.0","10.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790143366610432092","https://twitter.com/gwern/status/1790143366610432092","@SebastianB929 @ericzelikman This is actually far more creative than most splashy fonts. These differences are *enormous* compared to, say, the 'Chip' font you are reading now https://t.co/aSMHZ5dYAq as compared to America *or* Franklin Gothic: https://t.co/DVf5Xb2vRi","2024-05-13 22:13 +0000","134.0","14.0","0.1044776119402985","0.0","1.0","2.0","1.0","5.0","0.0","5.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790101840899534998","https://twitter.com/gwern/status/1790101840899534998","@teortaxesTex sorry i can't hear your attention tweets over my thoughts whirring about how OA can make a GPT-4o which is that much smarter than GPT-4-turbo, <200ms end-to-end voice<->voice latency, multimodal in text/image/sound, half the price of turbo, and available to free users too","2024-05-13 19:28 +0000","549.0","120.0","0.2185792349726776","0.0","2.0","25.0","11.0","0.0","0.0","72.0","0.0","0","0","0","0","0","10","10","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790097585237021175","https://twitter.com/gwern/status/1790097585237021175","@a_karvonen @DeveloperHarris @Teknium1 @sama All these discussions of how to handle interruptions without staggering waste sound an awful lot like ""just not requesting more than 2048 output tokens"" to me...","2024-05-13 19:11 +0000","208.0","14.0","0.0673076923076923","0.0","1.0","5.0","2.0","0.0","0.0","6.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790096643112513577","https://twitter.com/gwern/status/1790096643112513577","@Teknium1 @DeveloperHarris @sama ""my wife""","2024-05-13 19:07 +0000","215.0","21.0","0.09767441860465116","0.0","0.0","6.0","0.0","0.0","0.0","15.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790096507720327313","https://twitter.com/gwern/status/1790096507720327313","@DeveloperHarris @Teknium1 @sama How do you handle interruptions? Very common in conversation. Just throw away those 20,000 tokens you requested which are still streaming in?","2024-05-13 19:07 +0000","302.0","31.0","0.10264900662251655","0.0","2.0","10.0","5.0","0.0","0.0","14.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790095701910642804","https://twitter.com/gwern/status/1790095701910642804","@Teknium1 @sama Surprised to see no one mentioning my immediate assumption: latency. The longer the output, the more you potentially block, which defeats the point for interactive voice use. You don't need that many tokens all at once.","2024-05-13 19:04 +0000","2629.0","149.0","0.056675542031190566","0.0","2.0","54.0","29.0","0.0","0.0","64.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790095099747070317","https://twitter.com/gwern/status/1790095099747070317","@___frye Just tell him that it's like when Napoleon declared himself Emperor, or that other Emperor declared he was a human and not a god, or Lelouch manipulated Suzaku, or... wait, you said wife? Then it's like when Sunmi quit Wonder Girls.","2024-05-13 19:01 +0000","710.0","17.0","0.023943661971830985","0.0","0.0","10.0","4.0","0.0","0.0","3.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790091820921135414","https://twitter.com/gwern/status/1790091820921135414","@unixpickle You can still see the BPEisms, though. Like what is going on with those accents in the typewriter, or letters like 'l'/'i'?","2024-05-13 18:48 +0000","967.0","52.0","0.05377456049638056","0.0","0.0","13.0","9.0","0.0","0.0","27.0","0.0","0","0","0","0","0","3","3","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790070826215039289","https://twitter.com/gwern/status/1790070826215039289","@LiamFedus This seems like an increasing theme past few years in evaluations: certain models look similar on benchmarks full of relatively easy problems, but then the 'vibe' disagrees; later, we discover that they were overfit or do much worse on diverse/harder/more real-world problems.","2024-05-13 17:25 +0000","1008.0","187.0","0.18551587301587302","0.0","1.0","78.0","27.0","0.0","0.0","78.0","0.0","0","0","0","0","0","3","3","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1790050157217673244","https://twitter.com/gwern/status/1790050157217673244","@bae_theorem Also an example of how the Google Scholar index lags way behind the Google one. ?? I must've uploaded Seung et al 1992 years ago, and there's 3 PDFs in GS but none are mine: https://t.co/2Vp5a667D2","2024-05-13 16:03 +0000","487.0","14.0","0.028747433264887063","0.0","1.0","4.0","1.0","2.0","0.0","5.0","0.0","0","0","0","0","0","1","1","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789830416716763589","https://twitter.com/gwern/status/1789830416716763589","@teortaxesTex There are many attention mechanisms which try to shrink or approximate or compress down a lot, to something you might call a latent, because the sparsity of attention means you can throw away a lot of the full dense attention. Maybe I'm missing some implications here?","2024-05-13 01:29 +0000","296.0","41.0","0.13851351351351351","0.0","1.0","3.0","3.0","0.0","0.0","34.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789802337105543254","https://twitter.com/gwern/status/1789802337105543254","@yacineMTB https://t.co/U7gawtZXOy","2024-05-12 23:38 +0000","4333.0","595.0","0.13731825525040386","0.0","0.0","159.0","69.0","2.0","0.0","85.0","0.0","0","0","0","0","0","280","280","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789791863362248953","https://twitter.com/gwern/status/1789791863362248953","@palladiummag It's always San Francisco, isn't it? Or DC? No European or Chinese flies into Peoria & writes that...
I wonder how many billions of dollars of PR or international aid is undone annually the instant foreign elites fly into SF/DC, & see what those city governments want them to?","2024-05-12 22:56 +0000","218.0","22.0","0.10091743119266056","0.0","0.0","2.0","2.0","0.0","0.0","18.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789680665845186808","https://twitter.com/gwern/status/1789680665845186808","@nickcammarata But even lumping in everything like AoE2 campaign/Pokemon, when I think back to my collections by console, yeah, my clear rate was <10%. Once in a while I might make an effort to beat a specific game like _Halo_ on Legendary or _Ninja Gaiden Black_ on Hard, otherwise...","2024-05-12 15:34 +0000","1192.0","33.0","0.027684563758389263","0.0","0.0","2.0","2.0","0.0","0.0","29.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789680094321033255","https://twitter.com/gwern/status/1789680094321033255","@nickcammarata (Like, beating Pokemon, in the sense of beating Elite 4, is *so* easy that even Twitch Plays Pokemon ultimately did it. No one I knew took the slightest pride in beating it; you just had to in order to get Mewtwo. Getting all 151, now that was what beating Pokemon really was.)","2024-05-12 15:32 +0000","1428.0","49.0","0.03431372549019608","0.0","2.0","7.0","2.0","0.0","0.0","38.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789679617772581287","https://twitter.com/gwern/status/1789679617772581287","@nickcammarata Pokemon/Zelda/Ogre Battle are RPGs, AoE1 campaign isn't a game but a tutorial for multiplayer/online; Marios could be pretty hard (I beat Mario 64, but never NES Mario IIRC, the jumps just get too hard and takes too long to get to the killer gaps); dunno about the rest.","2024-05-12 15:30 +0000","615.0","33.0","0.05365853658536585","0.0","1.0","1.0","2.0","0.0","0.0","29.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789661092051906931","https://twitter.com/gwern/status/1789661092051906931","@nickcammarata There was never an era back then where you'd usually win a video game, though, outside RPGs. I played absurd amounts as a kid and I can count on one or two hands how many I ever won. It's post-2000 games which became obsessed with being user-friendly, completable, & non-grindy.","2024-05-12 14:17 +0000","772.0","32.0","0.04145077720207254","0.0","1.0","11.0","1.0","0.0","0.0","19.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789447656449663440","https://twitter.com/gwern/status/1789447656449663440","@JohnDCook https://t.co/P9AzsUqpzq","2024-05-12 00:09 +0000","752.0","52.0","0.06914893617021277","0.0","0.0","3.0","1.0","42.0","0.0","6.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789379266569818141","https://twitter.com/gwern/status/1789379266569818141","@krishnanrohit @KelseyTuoc ""many many decades""
We haven't even had them for one Japanese (ahem) life expectancy.","2024-05-11 19:37 +0000","180.0","24.0","0.13333333333333333","0.0","1.0","4.0","1.0","0.0","0.0","18.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789369740441407734","https://twitter.com/gwern/status/1789369740441407734","@krishnanrohit @KelseyTuoc Like, what price should global civilization pay to prevent ""the third and fourth worst firebombings of WWII"" from happening again...? Well, put that like that, not that much.
Specifically, not nearly as much as it ought to prevent ""nuclear war"".","2024-05-11 18:59 +0000","1090.0","28.0","0.025688073394495414","0.0","1.0","2.0","4.0","0.0","0.0","21.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789369104907780176","https://twitter.com/gwern/status/1789369104907780176","@krishnanrohit @KelseyTuoc I don't think bombing some civilians once or twice of a country starving to death which can't even shoot back at the bomber planes anymore (which is why it's going to unconditionally surrender a few days afterwards) really works for your rhetorical question here.","2024-05-11 18:56 +0000","1072.0","38.0","0.03544776119402985","0.0","1.0","1.0","2.0","0.0","0.0","34.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789360599295549784","https://twitter.com/gwern/status/1789360599295549784","@krishnanrohit @KelseyTuoc We did?
'Again'?","2024-05-11 18:23 +0000","231.0","10.0","0.04329004329004329","0.0","1.0","4.0","0.0","0.0","0.0","5.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789096678122741830","https://twitter.com/gwern/status/1789096678122741830","@patio11 ""Drop the 'the'. It's cleaner.""","2024-05-11 00:54 +0000","1952.0","71.0","0.03637295081967213","0.0","0.0","33.0","2.0","0.0","0.0","36.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789016449291276676","https://twitter.com/gwern/status/1789016449291276676","@mikeknoop @hyhieu226 Specifically, ""we should have evaluated with the objective of minimizing the estimated scaling law exponent, not minimizing loss at a fixed size on some arbitrary benchmark"".","2024-05-10 19:35 +0000","80.0","13.0","0.1625","0.0","0.0","4.0","0.0","0.0","0.0","9.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789014906911568250","https://twitter.com/gwern/status/1789014906911568250","@candrianillborn @L0m3z One of the more unexpected LW posts this year was Jessica Taylor writing a discussion/review of CTMU that was not entirely negative: https://t.co/Bhl4vYVfIv","2024-05-10 19:29 +0000","80.0","29.0","0.3625","0.0","0.0","2.0","0.0","21.0","0.0","6.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789003299066146848","https://twitter.com/gwern/status/1789003299066146848","@michael_nielsen The press release doesn't seem to say they were totally reliant on the external backups but seems to imply Google could recover most of it: ""...an extensive recovery of our Private Cloud which includes hundreds of virtual machines, databases and applications.""","2024-05-10 18:43 +0000","3602.0","131.0","0.036368684064408664","0.0","2.0","69.0","28.0","0.0","0.0","32.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1789002816633147412","https://twitter.com/gwern/status/1789002816633147412","@voogt_pieter @EigenGender https://t.co/WR32JB2F1T","2024-05-10 18:41 +0000","78.0","31.0","0.3974358974358974","0.0","0.0","2.0","1.0","12.0","0.0","16.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788979018890457232","https://twitter.com/gwern/status/1788979018890457232","@ChrSzegedy https://t.co/uzvJvLeKnw","2024-05-10 17:06 +0000","603.0","54.0","0.08955223880597014","0.0","0.0","8.0","1.0","31.0","0.0","14.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788970686473445633","https://twitter.com/gwern/status/1788970686473445633","@SilverVVulpes Disappointingly, OP says it's just a joke/meme: https://t.co/PiIajjwDMP Nevertheless, I choose to believe.","2024-05-10 16:33 +0000","645.0","34.0","0.05271317829457364","0.0","2.0","8.0","3.0","12.0","0.0","9.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788944104056840482","https://twitter.com/gwern/status/1788944104056840482","@iScienceLuvr Amazing how exactly people are reinventing Jason Antic's @DeOldify NoGAN approach, complete with perceptual losses for the finetuning: https://t.co/JgIjF629V8","2024-05-10 14:48 +0000","598.0","24.0","0.04013377926421405","0.0","0.0","3.0","3.0","12.0","0.0","6.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788932374467694950","https://twitter.com/gwern/status/1788932374467694950","@aepau2 @BarneyFlames @theshawwn helped work on The Pile with the rest of AI (specifically, books3), but not TWDNE (which was all me).
He did a lot of the GPT-2 stuff, did Tensorfork infrastructure and so enabled This Fursona Does Not Exist & This Anime Does Not Exist, and a bunch of other stuff.","2024-05-10 14:01 +0000","67.0","24.0","0.3582089552238806","0.0","0.0","2.0","1.0","0.0","0.0","21.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788750052472590503","https://twitter.com/gwern/status/1788750052472590503","@andrewwhite01 @FutureHouseSF https://t.co/KhXNJ6IsOl ""The DOI syntax shall be made up of a DOI prefix and a DOI suffix separated by a forward slash. There is no defined limit on the length of the DOI name, or of the DOI prefix or DOI suffix. The DOI name is case-insensitive and can incorporate any printable¡","2024-05-10 01:57 +0000","792.0","46.0","0.05808080808080808","0.0","1.0","4.0","4.0","2.0","0.0","35.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788744791057994060","https://twitter.com/gwern/status/1788744791057994060","(Every year I tweet mocking China AI hawks being wrong yet again, I get a frisson of fear¡ª""what if today is finally the day that they drop something as epochal as Vaswani or Brown or...?""
But then, a bird missed pooping on my head by a few feet yesterday.
To live is to risk.)","2024-05-10 01:36 +0000","2914.0","103.0","0.03534660260809883","0.0","1.0","22.0","4.0","13.0","0.0","63.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788731905212399832","https://twitter.com/gwern/status/1788731905212399832","@RebeccaSear The cells in my body all have somatic mutations, starting from conception, building up yearly as cell lineages proliferate & diversify & often spread body-wide. Then there are commensals, viral infection remnants, neuron self-splicing etc.
Is my body 'not a biological entity'?","2024-05-10 00:44 +0000","783.0","46.0","0.05874840357598978","0.0","0.0","32.0","7.0","0.0","0.0","7.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788729236192157865","https://twitter.com/gwern/status/1788729236192157865","@zraytam No one in ML knows any history pre-2020.
You think 2017 scaling law is amazing? There was a whole huge debate about decision trees vs logistic regression in the '90s with learning curves.
And if that blows your mind, wait til you see Highleyman in 1960: https://t.co/el2ZZwindb","2024-05-10 00:34 +0000","460.0","50.0","0.10869565217391304","0.0","0.0","8.0","1.0","35.0","0.0","6.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788725802961441149","https://twitter.com/gwern/status/1788725802961441149","@norvid_studies Mostly discontinued. I still submit to the subreddit, though, but not as actively. Still haven't decided whether I want to try to bring it back to full activity & catch up, send out what I have & declare bankruptcy & resume, or just definitively declare it ended.","2024-05-10 00:20 +0000","135.0","17.0","0.1259259259259259","0.0","1.0","4.0","0.0","0.0","0.0","12.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788724525233832231","https://twitter.com/gwern/status/1788724525233832231","""Western researchers can't compete with their access to databases; they can't even read the important research papers all written in Mandarin or using services inaccessible past GFW. This shows the inherent advantage of authoritarian technocracy over decadent liberal democracy!""","2024-05-10 00:15 +0000","4064.0","90.0","0.02214566929133858","0.0","1.0","21.0","5.0","6.0","0.0","57.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788723380998349290","https://twitter.com/gwern/status/1788723380998349290","It's funny to imagine the alternate history, where Dario Amodei still works for Baidu as a minor PM in the ERNIE division, where American AI is in shambles ""because FANG is too short-sighted to invest, privacy rights too strong, and Western Internet too hopelessly fragmented.""","2024-05-10 00:11 +0000","1810.0","39.0","0.02154696132596685","0.0","1.0","11.0","0.0","6.0","0.0","21.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788722946103533677","https://twitter.com/gwern/status/1788722946103533677","There was a year where the Chinese DL giants had developed rapidly for years before, and citation rates were surpassing American, and it looked like they were going to accelerate into DL scaling hyperspace, leaving the West behind & handing it all to Xi.
That year was... 2018.","2024-05-10 00:09 +0000","1252.0","67.0","0.05351437699680511","0.0","2.0","20.0","4.0","0.0","0.0","41.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788722353221812225","https://twitter.com/gwern/status/1788722353221812225","Or how about in 2017, when Baidu researchers published a little thing you might have heard of since then, called ""scaling laws""? https://t.co/RM4F8CVfT3","2024-05-10 00:06 +0000","1114.0","82.0","0.07360861759425494","0.0","1.0","13.0","2.0","49.0","0.0","17.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788722053954048475","https://twitter.com/gwern/status/1788722053954048475","BTW, one of the funniest things about my challenge here is that it gets easier the further back you go. Like FlashAttention's careful GPU memory hierarchy use allowing much greater NN scaling & speed... you know, like Baidu's 2016 Persistent RNNs (https://t.co/eAQJbRepv3).","2024-05-10 00:05 +0000","812.0","45.0","0.05541871921182266","0.0","1.0","8.0","4.0","15.0","0.0","17.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788639097575067652","https://twitter.com/gwern/status/1788639097575067652","@michael_nielsen ""I could tell you something."" https://t.co/E8lkcEJ3z9","2024-05-09 18:36 +0000","1105.0","109.0","0.09864253393665158","0.0","1.0","2.0","4.0","56.0","0.0","46.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788624583102169200","https://twitter.com/gwern/status/1788624583102169200","@Ted_Underwood Testing the following passage: another refusal, and the wording changes have become interesting.
Note how Claude-3 Opus silently switched ""revenge"" to ""glory"", which is a much more Anglo-Saxon motive for contending against the ""high king""... https://t.co/xJg1BTSG4S","2024-05-09 17:38 +0000","624.0","46.0","0.07371794871794872","0.0","1.0","2.0","3.0","0.0","0.0","21.0","0.0","0","0","0","0","0","19","19","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788623455669072159","https://twitter.com/gwern/status/1788623455669072159","@cosine_distance You can use it to entertain your cat.","2024-05-09 17:33 +0000","898.0","23.0","0.025612472160356347","0.0","0.0","14.0","6.0","0.0","0.0","3.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788616669536894999","https://twitter.com/gwern/status/1788616669536894999","@Ted_Underwood Claude-3 Opus: not bad results, but I'm more interested in its refusal to continue! That's unusual, and important: suggests that I can add a stopping command and it may terminate at a good point. https://t.co/U3b3v6nJwX","2024-05-09 17:06 +0000","441.0","27.0","0.061224489795918366","0.0","1.0","0.0","2.0","0.0","0.0","10.0","0.0","0","0","0","0","0","14","14","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788587861039546699","https://twitter.com/gwern/status/1788587861039546699","@main_horse @cto_junior That sounds kinda meh, but surely that emphasizes the point that they were probably just too busy to go into music in a big way?","2024-05-09 15:12 +0000","189.0","22.0","0.1164021164021164","0.0","1.0","2.0","0.0","0.0","0.0","19.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788581164334117029","https://twitter.com/gwern/status/1788581164334117029","@cto_junior @main_horse They seem very busy handling demand for their voice stuff already.","2024-05-09 14:45 +0000","188.0","8.0","0.0425531914893617","0.0","1.0","1.0","0.0","0.0","0.0","6.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788376875137417318","https://twitter.com/gwern/status/1788376875137417318","@rosstaylor90 @natfriedman","2024-05-09 01:14 +0000","532.0","45.0","0.08458646616541353","0.0","0.0","3.0","4.0","0.0","0.0","38.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788343200647708864","https://twitter.com/gwern/status/1788343200647708864","@moultano No, we're reacting to the convergent instrumental drives, like ""not being killed for human convenience and taking actions to do so"". A solitary orangutan or octopus is a genius escape artist, nothing social about it, and we come into conflict with them too, and same deal.","2024-05-08 23:00 +0000","193.0","26.0","0.13471502590673576","0.0","1.0","9.0","0.0","0.0","0.0","16.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788337086690472046","https://twitter.com/gwern/status/1788337086690472046","@GreatKingCnut @Algon_33 (I dislike touchscreen/tablet generally on principle, so I never even considered the Yoga Lenovos.)","2024-05-08 22:36 +0000","70.0","16.0","0.22857142857142856","0.0","1.0","1.0","0.0","0.0","0.0","14.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788335544675471770","https://twitter.com/gwern/status/1788335544675471770","@moultano You might not think crows are smart, until you kill one in sight of the others, and decades and multiple raven generations later, the murders are still dive-bombing you the moment you come near them. You may not use the word 'intelligence', but you see there's something there.","2024-05-08 22:29 +0000","652.0","36.0","0.05521472392638037","0.0","1.0","17.0","1.0","0.0","0.0","17.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788335157688037734","https://twitter.com/gwern/status/1788335157688037734","@moultano I think we could. Look at dolphins, orangutan / octopus escapes, elephants, the smarter birds like parrots, raccoons, bees etc. Even where we don't study them, we can tell they're smart.
The anthropocene sees animal intelligence as attacks, & routes around them or attacks back.","2024-05-08 22:28 +0000","689.0","31.0","0.04499274310595065","0.0","2.0","13.0","2.0","0.0","0.0","12.0","0.0","0","0","0","0","0","2","2","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788333046707458556","https://twitter.com/gwern/status/1788333046707458556","@seekingyaga @Algon_33 @ID_AA_Carmack What we find pleasant & motivational & esthetic doesn't necessarily affect our performance. (Look at those submarine studies - a submarine is not a nice place or somewhere to take a vacation.) Consider music. People swear by it, yet, little or harmful effects on cognition.","2024-05-08 22:19 +0000","55.0","3.0","0.05454545454545454","0.0","1.0","0.0","0.0","0.0","0.0","2.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788332537447657529","https://twitter.com/gwern/status/1788332537447657529","@Algon_33 Well, I'm not that ripped, although after lugging that thing around London and SF all these years, I ought to be...
I'm probably going to buy a Thinkpad Carbon for travel. (The HP Dragonfly looks wonderful, with that square screen, but I'm not sure I can ever trust HP again.)","2024-05-08 22:17 +0000","78.0","6.0","0.07692307692307693","0.0","1.0","0.0","2.0","0.0","0.0","3.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788330680469913646","https://twitter.com/gwern/status/1788330680469913646","@Algon_33 It's just what the Lenovo P70 Thinkpad had for GPU. It's actually still remarkably usable despite me buying it in 2017, because I went for 32GB RAM & replaced the hard drive with an SSD, once I plug it into a proper monitor & keyboard. (It's very heavy, but NBD here.)","2024-05-08 22:10 +0000","110.0","5.0","0.045454545454545456","0.0","1.0","2.0","0.0","0.0","0.0","2.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788325701168386488","https://twitter.com/gwern/status/1788325701168386488","@Algon_33 Ubuntu Firefox can't open files in /tmp/ anymore 'because Snap'. Nvidia Quadro GPUs a *decade+ later* color the screen green. Ubuntu 24 NPM package so outdated it crashes before even running `--help`. Latest Haskell regex libraries crashes on curly brackets, and Haskell version¡","2024-05-08 21:50 +0000","537.0","30.0","0.055865921787709494","0.0","1.0","4.0","1.0","0.0","0.0","23.0","0.0","0","0","0","0","0","1","1","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788325188284596655","https://twitter.com/gwern/status/1788325188284596655","@Algon_33 Yeah, data loss wasn't the issue. (Although I did need ChatGPT to figure out all the LVM commands to mount the old NVM drive's LVM/LUKS encryption.) It was everything else. Stunning BS: Ubuntu no longer offers a DVD install image (!). Debian testing corrupts itself with¡","2024-05-08 21:48 +0000","526.0","35.0","0.06653992395437262","0.0","1.0","2.0","1.0","0.0","0.0","31.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788323801769369996","https://twitter.com/gwern/status/1788323801769369996","@Algon_33 @ID_AA_Carmack Among other things... my workstation died Friday night, and after an absurd amount of yakshaving, I've only just reached the point of being able to update https://t.co/2d1yqGzqzF again on my backup laptop. ?","2024-05-08 21:43 +0000","89.0","8.0","0.0898876404494382","0.0","1.0","5.0","0.0","0.0","0.0","2.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788322003054649687","https://twitter.com/gwern/status/1788322003054649687","@Algon_33 @ID_AA_Carmack That's fine. I should write more about it but meh.","2024-05-08 21:36 +0000","89.0","8.0","0.0898876404494382","0.0","1.0","3.0","1.0","0.0","0.0","3.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788321774561575115","https://twitter.com/gwern/status/1788321774561575115","@idavidrein The Udio guys left DM relatively recently & DM hasn't done much music-related (that was GB). The media coverage of Suno mentioned they only incorporated in like, mid-2023 or something? So they launched soon after that. None of them seem like they've been grinding since April 2020","2024-05-08 21:35 +0000","150.0","3.0","0.02","0.0","0.0","3.0","0.0","0.0","0.0","0.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788321090135011731","https://twitter.com/gwern/status/1788321090135011731","@Algon_33 @ID_AA_Carmack Yeah, at the time I thought someone had screwed up, & it seemed like the old military studies might've been bad and highly motivated to null.
But then I got a look at the description of the Satish cognition benchmark, other Satish studies like the blinds, the chess null, etc.","2024-05-08 21:32 +0000","87.0","4.0","0.04597701149425287","0.0","1.0","1.0","0.0","0.0","0.0","2.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788320342122832321","https://twitter.com/gwern/status/1788320342122832321","@Algon_33 @ID_AA_Carmack So at this point, I am concerned that Satish, who seems to be the common factor to all of these anomalously multi-SD-outlier environmental effects, is simply another Ariely or Gino or Rosenthal... And if you exclude anything which Satish might have touched, CO2 isn't important.","2024-05-08 21:29 +0000","396.0","27.0","0.06818181818181818","0.0","0.0","18.0","1.0","0.0","0.0","8.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788319371351171434","https://twitter.com/gwern/status/1788319371351171434","@Algon_33 @ID_AA_Carmack There have also been some drastic failures to replicate, like https://t.co/uW7sDxgfaQ is a huge null that should be totally impossible if Satish/Allen-level effects were real.","2024-05-08 21:25 +0000","432.0","29.0","0.06712962962962964","0.0","1.0","9.0","1.0","7.0","0.0","11.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788318895591268465","https://twitter.com/gwern/status/1788318895591268465","(This overhang seems to have been mostly eaten up by the rash of music DL startups like Suno and Udio.
They don't seem to have any secret weapon like copyright or some brilliant new arch, so I guess it was just an ordinary 'automation as colonization wave' delay after all?)","2024-05-08 21:23 +0000","2809.0","73.0","0.025987896048415806","0.0","3.0","19.0","7.0","1.0","0.0","43.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788318229787537640","https://twitter.com/gwern/status/1788318229787537640","@Algon_33 @ID_AA_Carmack https://t.co/xjrsLHbaEN","2024-05-08 21:21 +0000","639.0","17.0","0.026604068857589983","0.0","1.0","8.0","1.0","0.0","0.0","7.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788317812252975401","https://twitter.com/gwern/status/1788317812252975401","@Algon_33 @ID_AA_Carmack See my other comments on that page. I hadn't seen the very extensive submarine literature at very high concentrations yet, and looking closer at Satish and Allen and the proprietary tests used (which required book scans etc), I became more and more suspicious.","2024-05-08 21:19 +0000","482.0","35.0","0.07261410788381743","0.0","2.0","13.0","2.0","1.0","0.0","17.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788315015939805558","https://twitter.com/gwern/status/1788315015939805558","@Algon_33 @ID_AA_Carmack sus","2024-05-08 21:08 +0000","301.0","43.0","0.14285714285714285","0.0","3.0","23.0","1.0","0.0","0.0","14.0","0.0","0","0","0","0","0","2","2","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788289727268802631","https://twitter.com/gwern/status/1788289727268802631","@Teknium1 https://t.co/JHwvsXZFzL is old news, but it's impossible to be COVD-19-related because the SAPA IQ data here ended 2018. So it's just whatever reverse Flynn effects are.","2024-05-08 19:27 +0000","476.0","27.0","0.05672268907563025","0.0","0.0","6.0","4.0","8.0","0.0","9.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788258431863865704","https://twitter.com/gwern/status/1788258431863865704","@iamaheron_ https://t.co/erKsxbOjIG","2024-05-08 17:23 +0000","220.0","67.0","0.30454545454545456","0.0","1.0","8.0","1.0","37.0","0.0","19.0","0.0","0","0","0","0","0","1","1","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788257189078904895","https://twitter.com/gwern/status/1788257189078904895","@marktenenholtz Every time I think I've seen the worst tokenizers have to offer...
How on earth did the T5 tokenizer drop curly braces?! Those aren't rare at all, even in web crawls filtered for language rather than code.","2024-05-08 17:18 +0000","648.0","40.0","0.06172839506172839","0.0","1.0","8.0","6.0","0.0","0.0","25.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788237422142066860","https://twitter.com/gwern/status/1788237422142066860","@visakanv Yeah, it's not that hard. This is just the WP API; it's free, you just call `https://t.co/vFd8ELk5h9` with a WP article title.
Our old simple JS version: https://t.co/6jiPOiKI8w
The problem is, it's very much WYSWIG there. No recursion, no links, nothing. If you want more...","2024-05-08 15:59 +0000","893.0","126.0","0.14109742441209405","0.0","0.0","61.0","9.0","33.0","0.0","23.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788235803983815068","https://twitter.com/gwern/status/1788235803983815068","@brain_exe_ai Yes. But it's not carefully ablated or anything, so don't feel you have to copy it or that it's even guaranteed to help any.","2024-05-08 15:53 +0000","642.0","17.0","0.0264797507788162","0.0","0.0","0.0","8.0","0.0","0.0","9.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788214696908136581","https://twitter.com/gwern/status/1788214696908136581","@perceptions420 No.","2024-05-08 14:29 +0000","63.0","18.0","0.2857142857142857","0.0","0.0","2.0","1.0","0.0","0.0","15.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1788204398549631041","https://twitter.com/gwern/status/1788204398549631041","@perceptions420 Yes. Although the past few years I have been trying out nicotine oral sprays (faster & more convenient package than gum, while still not going as far as vaping), which has been fine.","2024-05-08 13:48 +0000","58.0","12.0","0.20689655172413793","0.0","1.0","3.0","2.0","0.0","0.0","6.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1787997799738470877","https://twitter.com/gwern/status/1787997799738470877","@arankomatsuzaki I understand HuggingFace may have an exciting proposition for you.","2024-05-08 00:07 +0000","1300.0","103.0","0.07923076923076923","0.0","2.0","12.0","24.0","0.0","0.0","65.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1787960994515611824","https://twitter.com/gwern/status/1787960994515611824","@gojomo @AndrewCurran_ It's why his ""I believe Microsoft will live up to its contracts"" quote is as ironic as the much more famous ""I think it's important that I can be fired"".","2024-05-07 21:41 +0000","120.0","20.0","0.16666666666666666","0.0","0.0","8.0","1.0","0.0","0.0","11.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1787956468605751623","https://twitter.com/gwern/status/1787956468605751623","@gojomo @AndrewCurran_ Remember, one of the threats Altman used to get back in charge of OA was that MS would just take away its compute. Apparently MS was ahead of contractual delivery obligations (bc demand) so it wouldn't even technically break their contract. And there would be no more, of course.","2024-05-07 21:23 +0000","581.0","26.0","0.04475043029259897","0.0","1.0","4.0","3.0","0.0","0.0","18.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1787955995836301665","https://twitter.com/gwern/status/1787955995836301665","@gojomo @AndrewCurran_ Yep. Once you reach IKEA or Hershey levels, things get weird.
And of course, this is exactly why MS is both trying to make itself indispensable to OA and make OA dispensable to MS. ""Ofc you can declare AGI, and we will live up to the letter of all our contracts... nothing else.""","2024-05-07 21:21 +0000","586.0","32.0","0.05460750853242321","0.0","1.0","6.0","2.0","0.0","0.0","23.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1787847298103345418","https://twitter.com/gwern/status/1787847298103345418","@menhguin @teortaxesTex Are you saying you aren't using Baidu Ernie every day while OA teeters on the brink of bankruptcy as a year and a half later, the PRC keeps accelerating into AI hyperspace, leaving the West far behind, thanks to their graduating millions of STEM engineers a year? Baffling. ?","2024-05-07 14:09 +0000","73.0","34.0","0.4657534246575342","0.0","0.0","3.0","1.0","0.0","0.0","30.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1787659733362479262","https://twitter.com/gwern/status/1787659733362479262","@norvid_studies @shinboson Looks like it, yeah - which makes sense given moa's tiny, tiny lil' heads: https://t.co/LLiowUHlgU https://t.co/h1sQtvkf6l","2024-05-07 01:44 +0000","104.0","23.0","0.22115384615384615","0.0","0.0","3.0","0.0","5.0","0.0","15.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1787658539831296511","https://twitter.com/gwern/status/1787658539831296511","@teortaxesTex At least it's not another 'we lied about cloning OA API outputs' LLM but something genuinely interesting. That said, MAML doesn't seem like it'd produce really robust minima... I'd expect this obfuscation to fall to something like high LR line searches or something.","2024-05-07 01:39 +0000","875.0","51.0","0.05828571428571429","0.0","0.0","17.0","5.0","0.0","0.0","29.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1787515185671922171","https://twitter.com/gwern/status/1787515185671922171","@AndrewCurran_ (They've always been in a frenemy cold war, it just went hot once Altman got back in control of OA and started the postmortem.)","2024-05-06 16:10 +0000","1209.0","49.0","0.04052936311000827","0.0","1.0","15.0","5.0","0.0","0.0","28.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1787483808830247341","https://twitter.com/gwern/status/1787483808830247341","@JGoldenGardiner @michael_nielsen The computational constraints bind all the way down! https://t.co/RnwcwNtcbp","2024-05-06 14:05 +0000","69.0","26.0","0.37681159420289856","0.0","0.0","1.0","0.0","16.0","0.0","9.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1787308700719259896","https://twitter.com/gwern/status/1787308700719259896","@bcjordan @michael_nielsen We can only pray that by being so well-funded, they'll be too distracted doing secret proprietary startup R&D to reproduce themselves (via grad students), and 'interest in quantum mechanics' will be selected out of the relevant cultural environment before it is too late. ?","2024-05-06 02:29 +0000","153.0","12.0","0.0784313725490196","0.0","0.0","6.0","0.0","0.0","0.0","6.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1787306553738924471","https://twitter.com/gwern/status/1787306553738924471","@michael_nielsen The real reason, ofc, is that the universe is defined elegantly by QM but actually computed classically. Once people figure out QC, they inevitably increase the compute load massively, OOMing the universe. So the simulators are forced to tweak the hyperparameters against QC...","2024-05-06 02:21 +0000","1899.0","85.0","0.04476040021063718","0.0","2.0","31.0","5.0","2.0","0.0","45.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1787299737495883892","https://twitter.com/gwern/status/1787299737495883892","@michael_nielsen Unapologetic price discrimination via unbundling seems like a big part of the puzzle.","2024-05-06 01:53 +0000","1427.0","32.0","0.02242466713384723","0.0","0.0","7.0","4.0","0.0","0.0","21.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1787293544043991158","https://twitter.com/gwern/status/1787293544043991158","@SkyLi0n Anna's Archive Worldcat dump?","2024-05-06 01:29 +0000","750.0","10.0","0.013333333333333334","0.0","0.0","3.0","0.0","0.0","0.0","7.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1786942746478965029","https://twitter.com/gwern/status/1786942746478965029","@garytaubes https://t.co/uPO17T5K0T","2024-05-05 02:15 +0000","97.0","17.0","0.17525773195876287","0.0","0.0","0.0","0.0","9.0","0.0","8.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1786412816158658923","https://twitter.com/gwern/status/1786412816158658923","@cHHillee Looks like it crossed >70% around September 2023? https://t.co/gbyz6g02l3 so ~1.3 years ahead of schedule","2024-05-03 15:09 +0000","243.0","28.0","0.11522633744855967","0.0","0.0","9.0","1.0","2.0","0.0","16.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1786408497673802005","https://twitter.com/gwern/status/1786408497673802005","@AsdentAsdetrk @jd_pressman @ohabryka @VesselOfSpirit (Why do you think I spent so much time on char-RNN (eg. https://t.co/uFcYVVwgwK https://t.co/pfErGLrlS9 ) and DCGAN?)","2024-05-03 14:52 +0000","80.0","14.0","0.175","0.0","0.0","3.0","0.0","4.0","0.0","7.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1786408028960370965","https://twitter.com/gwern/status/1786408028960370965","@jskf__ @jd_pressman @ohabryka @VesselOfSpirit The NYer profile is really something: https://t.co/oicZdOdPZn
Imagine Geoff Hinton telling you that he'd like *a* GPU to get NNs to run on your CUDA and you blowing him off and saying ""you know what the future of GPUs is? simulating pizzas in ovens! that's the real money""","2024-05-03 14:50 +0000","79.0","19.0","0.24050632911392406","0.0","0.0","8.0","0.0","4.0","0.0","7.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1786406668697293211","https://twitter.com/gwern/status/1786406668697293211","@jd_pressman @VesselOfSpirit @ohabryka (If you search Google Scholar, you'll find that the phrase was not quite a neologism because there's some statistical physics or other random papers which use 'scaling hypothesis', but in extremely minor and totally unrelated ways, so it was there for the taking.)","2024-05-03 14:45 +0000","849.0","50.0","0.05889281507656066","0.0","0.0","9.0","3.0","0.0","0.0","38.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1786406331047415855","https://twitter.com/gwern/status/1786406331047415855","@jd_pressman @VesselOfSpirit @ohabryka Yeah, I had to coin the phrase in 2020 for my GPT-3 rant because there were plenty of discussions of the general idea going way back but no one had ever pinned it down as a quippy phrase, and I'd come to appreciate how critical those are.","2024-05-03 14:43 +0000","906.0","123.0","0.1357615894039735","0.0","1.0","21.0","2.0","8.0","0.0","91.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1786405514500284500","https://twitter.com/gwern/status/1786405514500284500","@nabla_theta @AndreTI @karpathy @hardmaru (Indeed. My point there is that the radiation-induced faults might be really weird and not representable by any nice differentiable approach the way dropout is - but then you just simulate the faults manually, and can always evolve a robust net, if you have to.)","2024-05-03 14:40 +0000","297.0","10.0","0.03367003367003367","0.0","0.0","1.0","3.0","0.0","0.0","6.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1786179198941360563","https://twitter.com/gwern/status/1786179198941360563","@AndreTI @karpathy @hardmaru (Considering the cost of aerospace hardware, it seems worthwhile to train as good a small robust model as you possibly can down on the ground, rather than just YOLOing the weights up there and hoping it works out.)","2024-05-02 23:41 +0000","681.0","27.0","0.039647577092511016","0.0","0.0","4.0","2.0","0.0","0.0","21.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1786170125411889298","https://twitter.com/gwern/status/1786170125411889298","@AndreTI @karpathy My concern there is that dropout would not be equivalent to how the matmuls would actually go wrong. NNs can learn robustness to specific noises (eg the old OpenAI floating point net or @hardmaru's weight-agnostic nets) but they can't be immune to every possible noise a priori.","2024-05-02 23:05 +0000","934.0","32.0","0.034261241970021415","0.0","3.0","6.0","4.0","0.0","0.0","19.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1786161695183802697","https://twitter.com/gwern/status/1786161695183802697","@thombo89 (Twitter is actually supposed to be showing my funny new Sanic thumbnail so I can check the margins look right, but of course, it must've cached the page from sometime earlier... ???)","2024-05-02 22:31 +0000","101.0","6.0","0.0594059405940594","0.0","0.0","0.0","2.0","0.0","0.0","4.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1786161255780163636","https://twitter.com/gwern/status/1786161255780163636","@karpathy The LLM should be radiation-hardened (like https://t.co/VAEwKHmbKk https://t.co/wEjjUVLAoQ ).
Probably you could do something like quantization-aware training, but with rad-like faults in the matmuls; you can evolve robustness if you can't figure out an appropriate backprop.","2024-05-02 22:30 +0000","2007.0","121.0","0.06028898854010962","0.0","1.0","29.0","6.0","49.0","0.0","36.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1786065107463688434","https://twitter.com/gwern/status/1786065107463688434","Test https://t.co/eGH8jrO30X","2024-05-02 16:07 +0000","2254.0","249.0","0.11047027506654836","0.0","1.0","9.0","2.0","211.0","0.0","26.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1785848624489832742","https://twitter.com/gwern/status/1785848624489832742","@michael_nielsen They thank god for the AI startups: ""we're the ???? hard tech VC play¡ªnot chatbot toys.""","2024-05-02 01:47 +0000","698.0","61.0","0.08739255014326648","0.0","1.0","7.0","6.0","0.0","0.0","47.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1785835173101154437","https://twitter.com/gwern/status/1785835173101154437","@michael_nielsen (Sometimes I think God invented quantum-computing startups to assuage AI startups' self-esteem issues over lack of revenue.)","2024-05-02 00:54 +0000","3352.0","108.0","0.032219570405727926","0.0","2.0","40.0","12.0","0.0","0.0","54.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1785772907785826567","https://twitter.com/gwern/status/1785772907785826567","@hentaikid @LucreSnooker There's no reason to believe that's secure.","2024-05-01 20:46 +0000","149.0","7.0","0.04697986577181208","0.0","0.0","1.0","0.0","0.0","0.0","6.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
"1785475805163991230","https://twitter.com/gwern/status/1785475805163991230","@genomerambler As opposed to Scott not being exposed to dying people as a doctor? https://t.co/SMrYgyV5vI","2024-05-01 01:06 +0000","1021.0","119.0","0.11655239960822723","0.0","0.0","28.0","4.0","65.0","0.0","22.0","0.0","0","0","0","0","0","0","0","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-","-"
He interrupted her. Close at hand is a stable where two beautiful ponies are kept. They are snowy white, and are consecrated to the goddess Ku-wanon, the deity of mercy, who is the presiding genius of the temple. They are in the care of a young girl, and it is considered a pious duty to feed them. Pease and beans are for sale outside, and many devotees contribute a few cash for the benefit of the sacred animals. If the poor beasts should eat a quarter of what is offered to them, or, rather, of what is paid for, they would soon die of overfeeding. It is shrewdly suspected that the grain is sold many times over, in consequence of a collusion between the dealers and the keeper of the horses. At all events, the health of the animals is regarded, and it would never do to give them all that is presented. On their return from the garden they stopped at a place where eggs are hatched by artificial heat. They are placed over brick ovens or furnaces, where a gentle heat is kept up, and a man is constantly on watch to see that the fire neither burns too rapidly nor too slowly. A great heat would kill the vitality of the egg by baking it, while if the temperature falls below a certain point, the hatching process does not go on. When the little chicks appear, they are placed under the care of an artificial mother, which consists of a bed of soft down and feathers, with a cover three or four inches above it. This cover has strips of down hanging from it, and touching the bed below, and the chickens nestle there quite safe from outside cold. The Chinese have practised this artificial hatching and rearing for thousands of years, and relieved the hens of a great deal of the monotony of life. He would not have it in the scabbard, and when I laid it naked in his hand he kissed the hilt. Charlotte sent Gholson for Ned Ferry. Glancing from the window, I noticed that for some better convenience our scouts had left the grove, and the prisoners had been marched in and huddled close to the veranda-steps, under their heavy marching-guard of Louisianians. One of the blue-coats called up to me softly: "Dying--really?" He turned to his fellows--"Boys, Captain's dying." Assuming an air of having forgotten all about Dick¡¯s rhyme, he went to his place in the seat behind Jeff and the instant his safety belt was snapped Jeff signaled to a farmer who had come over to investigate and satisfy himself that the airplane had legitimate business there; the farmer kicked the stones used as chocks from under the landing tires and Jeff opened up the throttle. ¡°Yes,¡± Dick supplemented Larry¡¯s new point. ¡°Another thing, Sandy, that doesn¡¯t explain why he¡¯d take three boys and fly a ship he could never use on water¡ªwith an amphibian right here.¡± Should you leave me too, O my faithless ladie? And years of remorse and despair been your fate, That night was a purging. From thenceforward Reuben was to press on straight to his goal, with no more slackenings or diversions. "Is that you, Robin?" said a soft voice; and a female face was seen peeping half way down the stairs. HoMElãñÔóÂÜÀ³ó
ENTER NUMBET 0016www.hpqibeng.com.cn
lianbangzg.com.cn
www.kqsyxb.com.cn
www.onuhje.com.cn
www.tuuujy.com.cn
rmchain.com.cn
www.vguc.com.cn
www.slchain.com.cn
www.sytgroup.com.cn
ryqyfs.com.cn