Stop Being So Dumb On Twitter

all right so yesterday it it appears that we had a huge explosion of the internet talking about requests per second and I think this is pretty amazing kind of some of the things that were stated on here and so I I just want to kind of like look through these and try to understand them uh it kind for me it all started right here and this is kind of where I left it and I thought this would be great to just look at and try to figure out what we're even looking at on the internet because it seemed like all the responses were kind of wild uh one thing that's kind of an interesting point of view is that there's an entire section of the internet we're going to call them anties where no matter what somebody says they will find a way to respond as negatively as possible to it don't be an ATI okay uh second off base camp did 500 and let's see 5,250 requests uh per second at its peak yesterday mean response time was 90 milliseconds which is pretty good that's very very fast the SLA on producing your list of list of movies at Netflix was 400 milliseconds last time I was there so this is really really good so call that needed 500 cores let's see so call that needed 500 cores at Max load if you skipped redundancy you could probably do that with three boxes each running a Z5 192 core AMD chip with room to spare servers have gotten crazy fast so this is very true CPUs are very very fast these days the new CPU stuff is extremely fast Casey and I we we did like what a hourong discussion on CPUs where he goes through all the the coolest stuff with Zen 5 they are extremely fast uh three boxes with lots of back ends uh what was the P95 response time he does not talk about the P95 response time so I don't know what the P95 response time but the p50 is really good 90 milliseconds is really really good now obviously there must be something kind of this must be time when he says uh the response time was 90 milliseconds this must be time on their servers not including any of the latency between computer like between computers which I think that makes sense right you we're not actually measuring the distance of you trying to travel out to somebody's computer we're just talking about within what he can what something with uh something in which he can control this is how fast it was going yeah not round trip yeah time to First Bite oh it did does it say that meantime uh mean response time oh is it is it really time to first bite it could be time to First Bite you're right mean response time it could it could very well be uh let's see even if you are let's see even if you are using a sled with more pedestrian 2x epyc 9454 setup you'd get 96 cores per server so you'd need just five of them for the load of course you still have to think about redundancy job running databases Etc but in terms of app compute it's very cheap that's all a base camp a business doing tens of millions of dollars in ARR uh I'd wager that 99% of B2B SAS doesn't do nearly as many RPS and could thus get away from even less compute in fact I'd be surprised if 95% couldn't be served off a single beefy server I generally agree with the statement but let me kind of give you like my kind of my general take on this which is there's kind of two camps right now that exist which I think is kind of silly uh the two camps that exist are Camp uh we'll call this one Camp one over here which is going to be like levels we'll call levels and dhh Camp 2 is going to be like versel and versel is going to have a certain curve that what you what you have in life uh my my assumption is Dax actually lives somewhere in the middle and he tries to give you the power to do all the scaling you want probably closer to this side than this side but just allows you to have all the engineering but we'll ignore Dax for now okay Dax get the hell out of here Dax get the Dax get out of here all right versell gives you the ability to have really cheap infinite scale but then when you infinite scale well it can get really expensive whereas with this approach you have a uniform bill but you have to pay for it in Dev time but I'm also not convinced hear me out on this one I am not convinced that this is Dev free time I don't know if you can trade this Dev time to manage three four machines a database and stuff like that is near is is not a perfect tradeoff to using this this type of service I use versell as the example it doesn't necessarily mean versell I'm just using it as an example and so I don't think that that's true I think there's some percentage of trade-off 20% maybe 15% of the time you can let go or maybe it's 50% of the time you don't have to spend but you do have to spend some amount of time managing this stuff 0 90% of apps don't need infinite scale I think 99% of apps don't need infinite scale right there's uh there's also admin costs which you're Outsourcing the cloud yeah that's what I mean that's why I'm saying uh all of that is that you have Dev and people running the things or you let somebody else run them uh there's an inherent difference between the resource required for you JS crud API reading some uh some squeal light with zero real customers and what is required for an API of a real business with thousands of customers also factual no I don't think I think we all agree with that I didn't know uh let's see I don't know how a company worth 10 million AR feels comfortable running with little to none of these Mechanics for resilience and Recovery needed for site reliability I don't know what you mean he he never said any of those things so this so this is one of those things that is always very surprising he does use terms right here redundancy right he uses terms running jobs in the background databases I think all those things are there it's just not infinitely there I think that's the tradeoff he's he's saying he makes which is we have a level of redundancy that we believe is comfortable for say uh 49 right oh my goodness nice nines maybe he has four nines of up time and that's okay maybe he doesn't need five9 of uptime and that's fine managing a database is very hard I agree managing databases is is most certainly difficult uh so squeal light is not used in real businesses uh sque light's used all over the place I think sque light's actually fantastic uh 49s is plenty to be let's be honest 49s is most of what people need and so I think this stuff is all kind of just very interesting just it's a very interesting world that we're currently living in cuz it feels like a dichotomy that has been set up where either you use one and you are good and you're and you have a business that can scale or you use the other and you're a fool and somehow I just don't feel like that's that's the case I'm not trying to tell people to go this route I think you should go to the route in which you can uh build the best application with the best air handling and the best uh non-happy path you can possibly build uh it's strange that there's not an affordable software deployment platform that provides four to five9 out of the box well it is like for I mean in some sense for sale is that it's just that once you scale really high it's really hard you got to you got to remember that when you scale really really when you actually have real scale either you're paying it for it for somebody else to manage it or you're paying for you to hire somebody and manage it for you uh so dhh must be the guy who did 100 startups with jQuery no no no no that's levels that's levels levels did with with this dhh is the Ruby guy and so and here we go so this is very interesting right here so let's kind of keep let's go into it so hopefully we're going to get something uh doesn't let's see doesn't really mean anything in terms of performance considering we don't know what each request is doing a single core can send eight uh to 10,000 messages per second on a simple node plus socket iio by the way socket IO is crazy uh setup obviously that's not what's done here let's see you can see exactly what it does by using basan this is this obviously isn't a hello world example one request May well be rendering 1,000 to-dos or showing you a calendar full of events this is a real SAS app requests rather than benchmarking jocking this is so Good by the way I think this Benchmark jocking that we've done is one of the most destructive ways to measure if something is good or bad in the universe okay nobody cares that your hello world can do 100,000 requests per second okay I I literally don't care cuz often those things that you're doing these really amazing bench requests for they happen over the course of say like 10 seconds you have no idea what garbage collection's doing you don't know what database calls and your ACN nature of all your requests are going to be doing as your as your event Loop say in node gets just jam-packed with all sorts of different response times how does that affect your 90 percentile your 95 percentile your 99 percentile response times it's just like such a vacuumed way to look at stuff and if you ever for any reason consider one thing better than another based off of a Hello World example you are being dumb okay or shall I say you are being inexperienced yeah hello it's it's not hello world it's hell world I let's see am not so impressed by Tech levels uh uses but uh by how he makes money with those sides yeah I mean you should be impressed by the tech he uses because it's simple let's see uh server side rendering those to-do lists and calendar events client side rendering could potentially reduce load times even further yeah but then you have to you I mean you still have to offload all of that into a client that can respond to it and all that it it takes lot of effort to do that this is a non-trivial amount of work uh let's see so Negative Nancy 90 milliseconds seems insane if local 5,000 requests over 5,000 cors I mean obviously that's not anything he just said this is what I mean by being an antie is that he clearly stated that base camp yesterday served Peak traffic 5,000 requests a second it's spread across 500 cores uh I mean Ruby is not fast Ruby is just it's it's it's not fast but like why why you got why you got to be such an antie you know what I mean obviously this is not local 5,000 request 90 milliseconds for response time is for the median is is fantastic uh it's not fast but in 2024 you can throw Hardware at it you can uh but fi also 500 cores for 5,000 requests per second is also not insane I mean the hard part I have with a lot of these things is I recently measured a react server components and I could not get it to exceed four requests per second on a larger amount of rendered items so it's like and that was local testing so it's I get it this is doing about 10 this is obviously doing 10 requests per second and this isn't doing it my my example was doing no database I was doing nothing nothing but just the basics here this is doing a lot of databases this is doing a lot of like actually rendering because rendering a calendar is is quite a bit of effort right let's see I I would consider that unacceptable 90 Mill like what is this anyone here thinks 90 milliseconds is unreasonable for a calendar to be resolved zero 90 milliseconds is good yeah 90 milliseconds what are these people on I see it's just such a strange way to put these things together it takes 80 milliseconds it takes 80 milliseconds to realize you stubbed your toe I just don't understand how people who have shipped zero products can criticize anything it's I call this the refs right um refs are people who sit on the sideline and watch a basketball game get played they know nothing of the lows of losing they know nothing of the highs of winning they just simply sit there and call things out and that's it they call and the worst part is that in basketball there's a defined set of rules in engineering there are no rules right there's there is absolutely no like ex like you can't like in basketball if you dribble and hold on to a ball you cannot dribble again it's a definite rule right you cannot cross that line with with an engineering I can choose Ruby I can choose rust I can choose go I can choose Noe I can literally use Pearl and so it's like there the rules are way different you know the rules and so do I okay and so there's just these refs that run around the sideline just being like tweet I've never done that but that's wrong I can tell right away I can just I can tell 90 90 milliseconds this is the worst actual worst no one literally uses Pearl literally the power grid of America runs on Pearl so yes people literally use Pearl all right buddy oh I was joking dude classic I was joking I was anyways okay so all right so let's look at some of let's I want to kind of look at some of the other things before we go on all let's see the new I could build this in a weekend just dropped I could run your entire sass with tens of millions in AR on my laptop the internet is amazing and more so uh than among the hello world Benchmark jockeys uh buddy you could accomplish this on a single system with fast enough storage and RAM so here's where these things kind of get difficult you theoretically could and what I mean by that is that if you got all the right people to get in there and highly optimize the application potentially write rewrite everything in Rust and have it as or rust or Zig or even potentially even go and and get it as perfectly tuned as possible you might you you could probably drop sign iFly the amount of cores you're asking to use my guess is that you could probably bring this down to 25 cores in a in in a proper language or in C right one that's designed to be super super fast but I think that's what I think that's the whole point of this which is that he's saying where is it where's the word I know he says the word uh luxury somewhere doesn't he that effectively what he's saying is that it's luxury the ability to be able to send what like to be able to use a language that is inherently inefficient and you can serve with relatively low ease a high amount of requests per second and make tens of millions of dollars off of hundreds of dollars a month in server costs did he say he needed 500 cores is that what he just said so that's another uh thing that people weren't looking uh with room to spare meaning that he's saying he could easily he could probably go higher I think that's what he's saying anyways uh I think in the end a lot of people don't realize the complication of building bigger apps uh you got to remember that base camp is also an application that has been running and being built over the course of a long period of time and and it's very easy to rebuild something in a very small way and show that it's fast it's really hard to build a whole application and show that it's fast and so it's a very interesting thing people took his Peak and Max load core number and put them together yes yes anyways I think I think this stuff is very very interesting because it's it's it's so it's so wild the backlash that both dhh and levels receives despite most people not making businesses as big or as profitable as either and the thing is if somebody came to me and said hey look at this I have a million dollar like like like Theo's upload thing I I have I I I give zero I give zero craps that it's ran off of nextjs it works right I'm not here to tell him how to live his life if that thing went from being from costing a little bit to costing him you know $100,000 a month I'm going to say well you you chose you you chose the wrong thing for whatever reason I I don't see it costing him that much just to due to how it was built but it's kind of it's kind of ridiculous like how much people get so worked up over each thing like I get zero work up about this and I get zero zero worked up about the opposite side the versel side like you can do whatever you want I just personally like the challenge of building something that's fast like something about that makes me feel good and that's I would call that the the The Artisan in me right the person that wants to that wants to build a really good product and some people want to build a really good product and they measure their goodness of the product say based on how much uh money they're making like levels IO right he's making products in which I would probably not call this an artisan product can we all agree to that and I think he would agree with it very very much so but his North Star is making money my North Star right now is just becoming really really good at a lot of Technology yeah the dopamine got you rush yeah uh you know you're around web devs and people start arguing about 2x performance improvements by changing the language it's well it's the there is virtually no other I mean there is there is some backend uh 1 million checkboxes talks about this which is the 1 million I guess you could still call this webd but having a server in Python and going from python to go was literally a Improvement but again that's just using a language that's better designed for the stuff we just live in a weird world in which we have the ability to choose a language in which is inherently inefficient for the tradeoff of feeling nice to program that's a really weird world because in like say the game development world you don't I mean you kind of do but almost any larger more like graphically intense game they don't have that choice they can't just be like I'm writing this graphically intense game in JavaScript like if you're painting some images on a canvas sure you can do that but like but but doing anything beyond that you're pretty much bound to C++ it's it's a weird world Indie G yeah Indie Dev games are injs yeah that's because you can because they're not doing something that's really intense any Indie uh Game Dev that's doing nontrivial stuff they're all built with unity they're all built with some other thing right and unity being probably one of the more big Hogs of memory and uh having a very difficult time scaling beyond that like tarov tarov has been notoriously known for poor performance and a lot of that has been just given to the fact that Unity is very difficult right gdau mentioned potentially yeah Lua is often used in Game Dev not for the game itself though there are some Lua is often used for the UI because it's really easy to embed into a C++ and very easy to be able to uh update and change because of the the the what's it called the dynamic nature of Lua does anyone think even think C++ and JS servers uh serve the same purpose yeah they don't and you should never do that the thing that makes JavaScript so great is the fact that you can provide a script and it runs there's no compilation there's no requiring certain libraries to be able to even have a compile on your system long as you either have node or it's in a browser it just runs it has its own sandboxed environment you're unlikely to get absolutely pwned through JavaScript I'm not saying it's not I'm just saying it's unlikely and so that's one thing that's really beautiful about JavaScript you have an application that is hard to update let's say an embedded device or something like that you use Lua you use a JavaScript one of these Dynamic languages you can update it by sending a new script there and then its UI can be slightly updated and you don't have to have somebody actually do a full update that's the beauty of JavaScript that's the beauty of Lua and that's really why I really like it a whole bunch is the ability to have that uh has Prime watch reacted to Jonathan Blow uh video saying games shouldn't use scripting languages like Lua I I just genuinely think he's wrong overall on that I think UI are best written in simple languages and you can do a lot of the Chrome really easy right game menus stuff like that I think that's just easier now the actual game itself can be all that but when you're like in matchmaking and you want a nice matchmaking like think about how many game options and servers and stuff like that are absolute horse garbage like just horse turds in a pasture most game menus are absolutely just the worst they take like 17 clicks to get through anything and so yeah throwing that crap in a little bit of Lua make it really easy to throw up a couple triangles and text and and check boxes it's just much nicer just to have something that's simple that you can go and update really easily right but the game menu has a swoosh how many days to learn Lua I'd say you could learn Lua if you're already familiar with the cbed language you could learn Lua in hour and a half if you really try not bad uh brenon Greg and I had an internal uh discussion at Netflix we started an internal podcast that became we are Netflix the internal podcast has a discussion between Brenan Greg and I and uh Lyall Troxel very very good I absolutely loved it use neovim and Lua is completed yeah yeah this is the reason right this is the reason why you like JavaScript I know I know a lot of people say they like to use typescript and all this stuff uh I I don't necessarily agree with this yet we use typescript to make those advantages go away I I don't think typescript makes all those advantages go away not at all uh not actually at all at all but the reason why you like typescript is very very simple let me show you let's pretend this was typescript for a second I go like this conu equals uh an object an array a string a number right like you can just create heterogeneous lists this is a very powerful powerful thing to do oh I want an object I just want properties I can just do that right that's a very powerful thing to do and it makes UI really easy to program because you don't have to have the expressed knowledge of everything you can actually you can just run with stuff very simple right and so I don't want to dog on it for that right B is one index yeah that's fine um again everything comes down to your ability to work with the language and you can do one index okay so anyways I wanted to talk about this RPS thing because I just mainly want people to stop being such anties you know if dhh chooses to do this like let me rephrase it this way dhh serves 5,000 requests per second with 90 milliseconds median time for base camp let's just say at 500 cores this costs him $11,000 a month it's like me having a discussion with my kid over his sister doing something that bothers him despite it not affecting him does this affect you no does this affect the world no does this mean anything to you no let him be happy about this okay cool bro makes bro makes 10 plus million dollars a year off of 500 CPU cores he employs people pays them money there's people that happily work at base camp and make a lot of money awesome just let it go because I am not going to write Ruby I'm gonna write go or rust or zigg or anything else because I can't do that cuz I feel emotionally bruised when I do that but that's up to him or PHP I'll write it in PHP okay easy as that just stop stop stop with the weird hate on the on the twitters move on have fun have fun with life a little bit more just if you can beat him then beat him build a better product if you think that he's truly doing it wrong like all these Andes in here that are just like I can't even believe this I believe this buddy you could accomplish this with a single system with fast enough storage Ram okay James build it better make it awesome because honestly if you can build a dramatically better product because you believe you can build a better product you will win the market you eventually will that's the beautiful part about SAS is that if you make a really good product you will start winning share the thing is I don't think you can I don't think I can do it either right I don't I don't think I could build a better calendar system in any reasonable amount less yappen more action let's go the name is the yappen