Always surprises me that they don't make specific content delivery systems for video, rather than rely on off-the-shelf parts. As they don't need general compute, but just video encode/decode and high raw bandwidth seems weird?

[insert picture of Homer Simpson drooling]

Stairmand:

Always surprises me that they don't make specific content delivery systems for video, rather than rely on off-the-shelf parts. As they don't need general compute, but just video encode/decode and high raw bandwidth seems weird?

Someone would need to design and build hardware for such specific workloads... and that would mean selling it in very limited quantities… it would be very expensive, probably not less expensive than the current solution.

True, but it's a growing market, and it's not like a Epic CPU and 250GB RAM setup is a bargain!

Stairmand:

True, but it's a growing market, and it's not like a Epic CPU and 250GB RAM setup is a bargain!

Nah, that's probably a bargain enough. Remember that it's for business. As long as your business generates revenue, which Netflix does, you can deduct investments from taxes. This is the huge difference between private consumers and business enterprises. Often businesses can't even afford to look for bargains because they need reliability. For example, a contract manufacturer in continental East Asia might make tools that would only last a month in serious use, but the factory itself requires quality tools to avoid production delays. The real question mark about the Netflix Epyc system would be the power use compared to specialised hardware.

not too long ago i was talking about Epyc's immediate market penetration in The Valley. i was laughed at. but being as the ones laughing didn't live and work in Silicon Valley as i do, i didn't hold their ignorance against them. yes Epyc trails Xeon for legacy based systems and in less demanding industries than computing. but in new systems and where high density, low power performance is required (like super-computers, streaming, pharmaceuticals, oil, and cloud, etc...) there is no substitute for Epyc nor will there be for the next five years.

Surprised they don't use Windows Server for that as its enterprise network drivers tend to be way better than *nix ones with better throughput at lower CPU use. I guess they don't want to pay for an OS.

Hilbert Hagedoorn:

A demonstration of Netflix's video streaming technology was held during the EuroBSD conference. That in and of itself isn't significant, but the company's optimizations in order to achieve 400 Gbit... Netflix operates on FreeBSD servers equipped with AMD Epyc CPUs and achieves roughly 400 Gbit/s per server.

Thanks for sharing this sort of content, it's awesome seeing the "behind the scenes" hardware details for major platforms.

Stairmand:

Always surprises me that they don't make specific content delivery systems for video, rather than rely on off-the-shelf parts. As they don't need general compute, but just video encode/decode and high raw bandwidth seems weird?

Google did it. Maybe Netflix don't have the talent and resources to design such a processor. Google develops custom 'Argos' video-transcoding chips for YouTube processing - DCD (datacenterdynamics.com)

MaCk0y:

Google did it. Maybe Netflix don't have the talent and resources to design such a processor. Google develops custom 'Argos' video-transcoding chips for YouTube processing - DCD (datacenterdynamics.com)

true dat. but they're also the single largest client for AMD Enterprise

richto:

Surprised they don't use Windows Server for that as its enterprise network drivers tend to be way better than *nix ones with better throughput at lower CPU use. I guess they don't want to pay for an OS.

The BSD network stack is one of the best in *Nix environment, even better than Linux's. Netflix did their homework, and it's not all about paying a licence, more to have something to tweak and modify to fit their needs.

richto:

Surprised they don't use Windows Server for that as its enterprise network drivers tend to be way better than *nix ones with better throughput at lower CPU use. I guess they don't want to pay for an OS.

Windows is terrible for this sort of thing. I did some high network processing work a while back, and even writing my own windows driver to bypass as much as Windows as possible you just can't get anywhere near the throughput of small packets like you can in Linux. Linux has some great libraries like DPDK (originally made by Intel) which has drivers specifically for Mellanox cards for getting wire rate transferes. I was able to receive 200Gbit UDP without dropping any packets on a single ConnectX-5 card on a Broadwell based server. Linux is just soo much more tweakable as well. You can completely remove certain CPU cores from the scheduler for running real-time threads on, as well as removing soo many components that you have a system barely interrupting your application. You just can't do anything like that in Windows. As anticupidon says, if you deploy on Windows Server you have to pay a license for each instance. Linux is free! That's a huge cost saving as well.

There are more reasons why TrueNAS, pFsense, Qnap and others choose BSD as their OS base. It may be lacking on the GPU part or desktop environment, but in the server platforms, BSD still has a place and a future.