As a front end developer, I am really happy to see resources like this.
Developing for the browser is a real challenge. I think working with html / css /js has been a neglected skill for a long time - most software engineers look down on that type of work and its rarely covered in comp sci course work.
Still, its good to see a lot of progress has been made, this book included.
My only critique - why use python instead of node.js?
Basically: server-side JavaScript is just not as widely known as Python, and it'd be additionally confusing when our browser starts running JavaScript. And in-browser JavaScript is a bit too restricted (by things like the same-origin policy) to do the whole thing inside a browser.
I see. I tried searching the table of contents as to "why python" and could not find it, but that link does more than enough to explain the "why."
I am resisting the urge to disagree, but since you did (literally) write a book about building a browser, I will defer to your expertise and try to learn from you :)
I can see that you'd want to pick a language that is good for teaching with, hence why you went with Python. It's probably too late, but I imagine Elixir would also be a great choice for this project.
I agree, JS _the language_ would be a more "obvious" choice for this, since it is both generally much faster and more popular in web development. I assume they are using Python _the ecosystem_ here. It probably comes with packages better suited for rendering specifically?
I don't think either is a super compelling choice anyways for this type of work. I think you want to use a systems language here. However Python is completely fine as a teaching language. Pretty much anyone who knows a similarly structured language can read it. And there is very little noise. So it can serve as a good reference if you want to follow along with a different language.
I think writing this in javascript would add some confusion, as people learning about this might have a hard time understanding exactly where the javascript is running.
> Developing for the browser is a real challenge. I think working with html / css /js has been a neglected skill for a long time - most software engineers look down on that type of work and its rarely covered in comp sci course work.
IMHO, those are job skills and not comp sci topics. They shouldn't be part of a degree program (except the most superficial treatment required to get some ugly UI up that may be required for something else). You have your whole career to pick them up.
Python seems like a good match since the tutorial seems to use what comes with a base python install. Python ships with Tk, for example, so the drawing section is cross-platform and not bogged down with a lot of instructions on adding new stuff with a package manager. That would detract from the lesson, IMO.
This made me start digging into whether this was considered a "safe" way of executing untrusted JavaScript in a sandbox.
It's not completely clear to me if DukPy currently attempts safe evaluation - it's missing options for setting time or memory limits on executed code for example: https://github.com/amol-/dukpy
I'm pretty paranoid though any time it comes to security and dependencies written in C, so I'd love to see a Python wrapper around a JavaScript engine that has safe sandbox execution as a key goal plus an extensive track record to back it up!
If you want battle-hardened, I figure you can't do better than V8. Here's a Python wrapper that I've poked at a bit (it's not quite 100% feature-complete but it seems to essentially work): https://github.com/sqreen/PyMiniRacer
I don't see any specific claims on isolation/memory safety or safety in general on duktapes page. Both V8 and spider monkey actively fix new JS vulnerabilities, and V8 isolates are used in the wild to success. Cloudflare workers is an example.
So far, my co-author Pavel has taught from this book multiple times (including this semester). In the spring at least one other university will offer a course. We'll list all known courses offerings on the website.
Also, if anyone would like to teach from this book, please get in touch!
Looking at the book's table of contents, I disagree. Browsers may resemble OSes by the size of the code base or by the amount of optimization involved, or by their importance for the modern world, but not by the types of technologies involved. I doubt the Browsers course will intersect a lot with OSes course.
EDIT: Reading you comment again I suspect you might have been joking :)
Looks like RemoteHQ launched something called Remote Browser as well, which is a SaaS browser, but more for collaborative purposes with more than one person using the same browser at the same time.
In my view, a critical part of really learning how something as complicated as a browser works is by trying to build it yourself. That's why our book is oriented around building a browser as you go.
This also looks great. Might you or anyone else know where to find the accompanying video? I get "Sorry
This video does not exist." for the embedded video link.
Many years ago, probably 20, I went on a task of implementing a web browser. I remember I gave up at rendering tables. I couldn't wrap my head around on how to properly size them. It has become extremely complex quickly to address edge cases and I eventually gave up when I couldn't understand what's going after having a two weeks break. Probably if I had money and was able to commit full time I could eventually get it, but I had to focus on commercial work and putting food on my table.
edit: it's a great article! But nothing on rendering tables :-)
> Many years ago, probably 20, I went on a task of implementing a web browser. I remember I gave up at rendering tables.
HTML 5 effort has cleaned up a lot of behaviors and specified how browser tags should behave. So it is, possibly, an approachable task now. Still daunting though.
Tables are still not that clean! But luckily tables are way way less important than they were in the past, so much so that browser differences in table rendering leaves most pages readable.
I'm the rendering lead for Chrome, and know quite a lot about how it works. I also recently wrote a series of articles about the new rendering architecture of Chromium, see here:
Pavel is a professor at the University of Utah and has extensively studied CSS from an academic point of view. He also has a lot of experience teaching the material and making it accessible to students.
This is awesome. I've always wanted to know how the actual layout portion works (or at least, can work in a simple way). I think these kinds of resources are really valuable and people should be empowered to make bespoke-ish web renderers as the need arises.
Author here. Layout is my favorite part of the browser—my dissertation is largely about formalizing browser layout—and so far the book only covers the basics, like the layout tree and how layout is computed with multiple tree traversals, but even understanding those basics I think gives you a huge advantage when thinking about web page layout tasks.
This book's browser supports JavaScript but the book itself skips over it because the browser just uses DuckPy. Which is a reasonable pedagogical choice. Building even a moderately "realistic" JavaScript interpreter would be its own semester-length course.
(This is a really fantastic resource; I'll probably use it as the basis of course once I get around to semi-retiring and taking a teaching job. Thanks, OP!)
It's now nearly impossible to build a web browser from scratch because of runaway explosion of web browser features, and proprietary API extensions.
W3C here is unfortunately a part to the problem.
Standardisation is good, but letting google pour streams halfassedly written RFCs onto other browsermakers is not good.
Non-enforcement of standards is also bad, and it's bad to extend W3C privileges to companies who themselves selectively implement their own proposals, so others' browsers can't match their behaviour.
"The total word count of the W3C specification catalogue is 114 million words at the time of writing. If you added the combined word counts of the C11, C++17, UEFI, USB 3.2, and POSIX specifications, all 8,754 published RFCs, and the combined word counts of everything on Wikipedia’s list of longest novels, you would be 12 million words short of the W3C specifications"
Author here—the browser the book works through is, uhh, pretty limited, so I don't imagine you'd want to actually use it for web browsing. It's more like writing your own toy compiler or operating system, to learn how they work.
As a front end developer, I am really happy to see resources like this.
Developing for the browser is a real challenge. I think working with html / css /js has been a neglected skill for a long time - most software engineers look down on that type of work and its rarely covered in comp sci course work.
Still, its good to see a lot of progress has been made, this book included.
My only critique - why use python instead of node.js?
Author here. I wrote up my answer here: http://browser.engineering/blog/why-python.html
Basically: server-side JavaScript is just not as widely known as Python, and it'd be additionally confusing when our browser starts running JavaScript. And in-browser JavaScript is a bit too restricted (by things like the same-origin policy) to do the whole thing inside a browser.
I see. I tried searching the table of contents as to "why python" and could not find it, but that link does more than enough to explain the "why."
I am resisting the urge to disagree, but since you did (literally) write a book about building a browser, I will defer to your expertise and try to learn from you :)
1 reply →
I can see that you'd want to pick a language that is good for teaching with, hence why you went with Python. It's probably too late, but I imagine Elixir would also be a great choice for this project.
This looks fantastic! Really excited to see this. I'm looking forward to reading the updates as well. Cheers.
Great blog design, btw! I love the way it displays footnotes (if you can call them that now) on page margins.
Does it have RSS?
1 reply →
I agree, JS _the language_ would be a more "obvious" choice for this, since it is both generally much faster and more popular in web development. I assume they are using Python _the ecosystem_ here. It probably comes with packages better suited for rendering specifically?
I don't think either is a super compelling choice anyways for this type of work. I think you want to use a systems language here. However Python is completely fine as a teaching language. Pretty much anyone who knows a similarly structured language can read it. And there is very little noise. So it can serve as a good reference if you want to follow along with a different language.
I think writing this in javascript would add some confusion, as people learning about this might have a hard time understanding exactly where the javascript is running.
> Developing for the browser is a real challenge. I think working with html / css /js has been a neglected skill for a long time - most software engineers look down on that type of work and its rarely covered in comp sci course work.
IMHO, those are job skills and not comp sci topics. They shouldn't be part of a degree program (except the most superficial treatment required to get some ugly UI up that may be required for something else). You have your whole career to pick them up.
Python seems like a good match since the tutorial seems to use what comes with a base python install. Python ships with Tk, for example, so the drawing section is cross-platform and not bogged down with a lot of instructions on adding new stuff with a package manager. That would detract from the lesson, IMO.
I was interested to see that this uses the DukPy wrapper around Duktape for the JavaScript interpreter: https://browser.engineering/scripts.html
This made me start digging into whether this was considered a "safe" way of executing untrusted JavaScript in a sandbox.
It's not completely clear to me if DukPy currently attempts safe evaluation - it's missing options for setting time or memory limits on executed code for example: https://github.com/amol-/dukpy
There's a QuickJS Python wrapper here which offers those limits: https://github.com/PetterS/quickjs
I'm pretty paranoid though any time it comes to security and dependencies written in C, so I'd love to see a Python wrapper around a JavaScript engine that has safe sandbox execution as a key goal plus an extensive track record to back it up!
If you want battle-hardened, I figure you can't do better than V8. Here's a Python wrapper that I've poked at a bit (it's not quite 100% feature-complete but it seems to essentially work): https://github.com/sqreen/PyMiniRacer
That looks really good - especially how they've managed to bundle a pre-compiled v8 into a 4MB Python wheel: https://blog.sqreen.com/embedding-javascript-into-python/
The time limit and memory limit support looks good too: https://github.com/sqreen/PyMiniRacer/blob/f7b9da0d4987ca7d1...
I don't see any specific claims on isolation/memory safety or safety in general on duktapes page. Both V8 and spider monkey actively fix new JS vulnerabilities, and V8 isolates are used in the wild to success. Cloudflare workers is an example.
I can't wait for Browser Engineering to show up as a university course a la compilers, operating systems, networks, etc.
Author here.
That's exactly what we are hoping for.
http://browser.engineering/preface.html
So far, my co-author Pavel has taught from this book multiple times (including this semester). In the spring at least one other university will offer a course. We'll list all known courses offerings on the website.
Also, if anyone would like to teach from this book, please get in touch!
A browser is basically an operating system inside an operating system.
The funny thing is we can have the MINIX microkernel discussions all over again :)
Looking at the book's table of contents, I disagree. Browsers may resemble OSes by the size of the code base or by the amount of optimization involved, or by their importance for the modern world, but not by the types of technologies involved. I doubt the Browsers course will intersect a lot with OSes course.
EDIT: Reading you comment again I suspect you might have been joking :)
7 replies →
Building a browser as a desktop application would be quite hard, but I reckon I do it as a web application.
Maybe we should have a SaaS browser.
s/ already exists: Mighty [https://news.ycombinator.com/item?id=26957215] which is $30-50 a month for remote session to Chrome running on a very powerful Ubuntu system.
Looks like RemoteHQ launched something called Remote Browser as well, which is a SaaS browser, but more for collaborative purposes with more than one person using the same browser at the same time.
I recommend this:
https://www.html5rocks.com/en/tutorials/internals/howbrowser...
Author here.
That article, along with a number of other resources, are listed here:
https://browser.engineering/bibliography.html
In my view, a critical part of really learning how something as complicated as a browser works is by trying to build it yourself. That's why our book is oriented around building a browser as you go.
This also looks great. Might you or anyone else know where to find the accompanying video? I get "Sorry This video does not exist." for the embedded video link.
Many years ago, probably 20, I went on a task of implementing a web browser. I remember I gave up at rendering tables. I couldn't wrap my head around on how to properly size them. It has become extremely complex quickly to address edge cases and I eventually gave up when I couldn't understand what's going after having a two weeks break. Probably if I had money and was able to commit full time I could eventually get it, but I had to focus on commercial work and putting food on my table.
edit: it's a great article! But nothing on rendering tables :-)
> Many years ago, probably 20, I went on a task of implementing a web browser. I remember I gave up at rendering tables.
HTML 5 effort has cleaned up a lot of behaviors and specified how browser tags should behave. So it is, possibly, an approachable task now. Still daunting though.
Tables are still not that clean! But luckily tables are way way less important than they were in the past, so much so that browser differences in table rendering leaves most pages readable.
Is this written by actual web browser engineers? If so, what fields did they specialize in?
Author here.
I'm the rendering lead for Chrome, and know quite a lot about how it works. I also recently wrote a series of articles about the new rendering architecture of Chromium, see here:
https://developer.chrome.com/blog/renderingng/
Pavel is a professor at the University of Utah and has extensively studied CSS from an academic point of view. He also has a lot of experience teaching the material and making it accessible to students.
https://mobile.twitter.com/chrishtr says "Rendering lead for Chrome", so at least one of the authors seems to do this professionally.
More about his team’s work: https://www.chromium.org/teams/rendering
This is awesome. I've always wanted to know how the actual layout portion works (or at least, can work in a simple way). I think these kinds of resources are really valuable and people should be empowered to make bespoke-ish web renderers as the need arises.
Author here. Layout is my favorite part of the browser—my dissertation is largely about formalizing browser layout—and so far the book only covers the basics, like the layout tree and how layout is computed with multiple tree traversals, but even understanding those basics I think gives you a huge advantage when thinking about web page layout tasks.
This video is good background for khtml/webkit on which chrome was based https://www.youtube.com/watch?v=Tldf1rT0Rn0
Awesome to see this here! The course that accompanied this textbook was among my favorites
There’s a 90’s style browser in Haskell here for demo purposes. Lacks JS.
https://github.com/chrisdone/vado
This book's browser supports JavaScript but the book itself skips over it because the browser just uses DuckPy. Which is a reasonable pedagogical choice. Building even a moderately "realistic" JavaScript interpreter would be its own semester-length course.
(This is a really fantastic resource; I'll probably use it as the basis of course once I get around to semi-retiring and taking a teaching job. Thanks, OP!)
Love the bibliography section. I have always wanted to reinterpret HTML into other representations. These resources give me good reference
It's now nearly impossible to build a web browser from scratch because of runaway explosion of web browser features, and proprietary API extensions.
W3C here is unfortunately a part to the problem.
Standardisation is good, but letting google pour streams halfassedly written RFCs onto other browsermakers is not good.
Non-enforcement of standards is also bad, and it's bad to extend W3C privileges to companies who themselves selectively implement their own proposals, so others' browsers can't match their behaviour.
Actually a from-scratch web browser is being built:
https://www.fastcompany.com/90611677/flow-ekioh-web-browser-...
Also you should take a look at the WHATWG because it’s far more relevant than the W3C nowadays.
"The total word count of the W3C specification catalogue is 114 million words at the time of writing. If you added the combined word counts of the C11, C++17, UEFI, USB 3.2, and POSIX specifications, all 8,754 published RFCs, and the combined word counts of everything on Wikipedia’s list of longest novels, you would be 12 million words short of the W3C specifications"
https://drewdevault.com/2020/03/18/Reckless-limitless-scope....
No idea how Flow does it, but building a browser is nearly impossible.
7 replies →
Flow browser isn't FOSS.
> WHATWG [is] far more relevant than the W3C nowadays.
Which is arguably part of the problem.
Love first principles stuff. Great job.
this builds a web browser like the build-your-own-x movement?
Author here—the browser the book works through is, uhh, pretty limited, so I don't imagine you'd want to actually use it for web browsing. It's more like writing your own toy compiler or operating system, to learn how they work.
It's perfect then! I'll read and work the exercises.
Thanks for answer!
Neat, but what about integrating widevine support?