What exactly is HTTP/3? Why was it needed so soon after HTTP/2 (which was only finalized in 2015)? How can or should you use it? And especially, how does this improve web performance? Let’s find out.
After almost five years in development, the new HTTP/3 protocol is nearing its final form. Earlier iterations were already available as an experimental feature, but you can expect the availability and use of HTTP/3 proper to ramp up over in 2021. So what exactly is HTTP/3? Why was it needed so soon after HTTP/2? How can or should you use it? And especially, how does it improve web performance? Let’s find out.
If you ask me to name the most misunderstood aspect of Python, I will answer without a second thought: the Python import system. Just remember how...
Victor who is working on the Python BTS (behind-the-scenes) series writes,
The Python import system doesn't just seem complicated – it is complicated. So even though the documentation is really good, it doesn't give you the full picture of what's going on. The only way to get such a picture is to study what happens behind the scenes when Python executes an import statement. And that's what we're going to do today.
PEP 636 introduced a new match expression which effectively is a switch statement on steriods. It not only fills the void due to the lack of switch cases in Python but caters to advanced use cases that will supercharge your Python codebase. Its basically Guido's way of saying, In you're face to the other programming languages community.
A simplest example would be to evaluate an expression and match it with the corresponding case from top-to-bottom. Once the match is found, it will execute the body of that case, and ignore the rest of the cases. It doesnt need a "break" or a default case like we have in switch cases of other languages.
match operation: case "add": result = a + b case "sub": res = a - b
The behavior is exactly like if/elif statements, albeit, a much more idiomatic syntax to please Python enthusiasts.
Advanced use-case
The match expression can be any Python statement, and the case statements can look for complex values. The interpreter can bind values with the case variables, if the sgnature matches. It can match subpatterns, allows using conditional expression, bind values in a dict, or use a wildchar _ to match anything.
match operation.split(): case ["add", a, b]: res = a+b case ["sub", *operands]: a, b = operands res = a-b case ["div", obj]: res = obj.a / obj.b case ["mul", a, b] | ["*", a, b]: res = a*b case [("^" | "xor"), a, b]: res = a^b case _: printf("Always matches, but doesn't bind any variables")
Release
PEP 636 will be released in the alpha channel of v3.10, and I'm excited to try it asap. Overall it looks really promising and intuitive, and with a few minor tweaks it should be good enough to be released in Python 3.10. For more detailed examples, please visit the official PEP Index.
In this article, we'll explore some of the basic concepts of Apache Kafka with hand drawn illustrations and cover all the commonly used terminologies in relation to Kafka.
A typical messaging system sends a message point-to-point from the sender to the receiver.
If the sender wants to send data to multiple receivers, it has to duplicate information and send it separately to each receiver.
Clearly this system doesn’t scale well and is grossly inefficient. This is where a publish-subscribe messaging system comes into the picture. In this system, the publisher sends message to a node to which one or more subscribers are listening to.
In the Kafka world, a publisher is called a Producer, and a subscriber is called a Consumer. In the real world, there can be one or more Producers (called a Producer group) sending messages to a node, and one or more Consumers (called a Consumer group) subscribing for messages from a node. A message in Kafka is called a Record. Each record is a byte-array that can store objects of any format.
A node in the case of Kafka is called a Broker, which has one or more Topics. Topics are categories to which Producer sends a message and consumer subscribes to. Each topic is then divided into Partitions so that multiple consumers can read from the same topic in parallel.
Each topic in the diagram has 2 partitions. A producer will send a record to a given Topic-Partition and a consumer will read a record from a Topic-Partition. Partitions are log of records and each new record is added at the end of the log. The consumer can decide from which offset in the logs, it wants to read the record.
Now having a single broker is not good for fault tolerance, if that Broker goes down. Hence, Kafka allows setting up multiple brokers. A group of brokers is called a Cluster. Management of brokers within a cluster, is performed by Zookeeper. There can be one or more clusters in a single Zookeeper instance.
In case of a multi broker setup, each broker will have a Topic → Partitions. These brokers are replicas of each other, with one being the leader, who has the responsibility of replicating records between partitions of other Brokers. If the Broker goes down, the replica will start operating as the leader. This allows us to build a scalable, distributed, and a fault-tolerant architecture.
For more information, read the official quick start guide of Apache Kafka.
In many ways Python's approach to handle references is quite different to other programming languages but I find it remarkably intuitive if you understand the idea of how everything in Python is an object and how objects are supposed to behave when passed as arguments to a function.
Lets start with a simplest example,
>>> a = 1 >>> def f(a): ... print("Inside func:", a) ... a = 2 ... >>> f(a) Inside func: 1 >>> a 1
In a regular programming language, we'd be quick to deduce that a in this case is passed by value to the function f, which then is overwritten inside the scope of the function which has no bearing on the variable a that exists outside of it. So far so good?
Well no, it'd have been nice to print the memory address alongwith the value, just to confirm this behavior.
>>> a = 1 >>> id(a) 1488123824 >>> def f(a): ... print("Inside func:", a, id(a)) ... a = 2 ... >>> f(a) Inside func: 1 1488123824 >>> a 1
Hang on! we thought that only the value is passed to the function, however, id(a) suggests that they are both indeed references to the same memory location. So is Python now doing pass by reference?
Not exactly! In order to understand how this works in Python, you'd have to get rid of all your pre-conceived notions of variable references in other programming languages and instead evaluate this behavior from a fresh perspective.
Everything is an object
Remember when I said that how everything in Python is an object? I wasn't kidding. In this case, a is a reference to an object of type integer with a value 1.
So when Python calls the function, it simply passes the reference to the integer object which is much more efficient than duplicating the same value (1) in two different memory locations, if it were to do pass-by-value.
Now when the interpreter encounters the assignment expression a = 2, Python creates a new reference within the local scope of the function that points to the memory location of the Integer object holding the value 2 and we lose the reference to the value 1. This shouldn't be surprising because Python behaves exactly like this even when you test this outside the function in a shell.
>>> a = 1 >>> id(a) 1488123824 >>> a = 2 >>> id(a) 1488123840
If you were to visualize, it would look something like this,
The behavior is exactly the same regardless of the type of an object, each time Python encounters an assignment expression, be it inside or outside the function.
Since list is a mutable data type, if we modify the list inside the scope of the function, it will continue to persist outside the scope because Python is basically working on the references to the existing memory location. In my view this behavior is consistent with how we expect it to work.
Hence, if were to condense the behavior under a single rule, it would read like this,
Python will continue to use the same references to objects until it encounters an assignment expression irrespective of the scope it is operating under. Python only creates new reference when it needs to, which makes Python's implementation of passing the parcel several orders of magnitude efficient than other languages.
So if next time someone asks, if Python is "pass-by-value" or "pass-by-reference", the answer is it depends.
The walrus operator (:=) is perhaps the most controversial operator in the history of Python. Guido was so unhappy with the proposal that he even stepped down permanently from his BDFL role after he accepted this operator as part of PEP 572.
As the name suggests, the walrus operator inherits its name from the two eyes and the teeth represented by the colon (:) and the equal (=) operators in conjunction. It is a higher order form of assignment expression designed to do two things at once,
Assign the variable
Return the value of the variable
Lets consider an example,
When you match a pattern for a regular expression, you save the result in a variable res which may or may not be None. You have to check for NoneTypes before retreiving the groups from the matched pattern.
>>> res = re.match(r'\s') >>> if res: ... print(res.group(0))
With the introduction of the walrus operator, you can now write the same code as,
The operator is used as part of an expression, so writing y:=f(x) is invalid. It has to be surrounded by parenthesis like, (y:=f(x)). This might be a bit wierd because generally Python doesnt enforce the usage of parenthesis as part of its design language. Beyond this caveat, there are few other criticisms with the introduction of this operator,
It is not so obvious to a beginner level programmer despite the existence of this operator in other languages.
It goes against the Zen of Python which describes, "There should be one—and preferably only one—obvious way to do it".
Apart from #2, it also disobeys the principle, "Explicit is better than implicit", "Simple is better than complex" by adding complexity in the name of saving whitespaces.
There's not enough real world usage data of how developers intend to use it.
Hate it or like it, the walrus operator is officially part of Python 3.8!
This post is about using the different assert methods available in the unittest framework. It is however, has no resemblance with the python assert statements.
Python's unittest framework hosts a bunch of useful assert methods that we can use to validate the behaviour of our tests. Before, we begin, it is essential to add the unittest framework to our code and setup the test class.
It accepts two elements to check for equality with an optional msg parameter which will print when the assertion fails.
>>> a, b = 1, 1 >>> tc.assertEqual(a, b)
>>> a, b = 1, 2 >>> tc.assertEqual(a, b, msg=f'a={a} and b={b} are not equal') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3.8/unittest/case.py", line 912, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.8/unittest/case.py", line 905, in _baseAssertEqual raise self.failureException(msg) AssertionError: 1 != 2 : a=1 and b=2 are not equal
Similarly, we have assertNotEqual to assert for inequality. assertEquals has been deprecated since.
assertTrue / assertFalse
It accepts an expressions that evaluates to a boolean value with an optional msg parameter which will print when the assertion fails.
>>> tc.assertTrue(bool(1)) >>> tc.assertTrue(bool(0), msg='0 is not a truthy expression') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3.8/unittest/case.py", line 765, in assertTrue raise self.failureException(msg) AssertionError: False is not true : 0 is not a truthy expression
assertIs / assertIsNot
It checks for the identity of the two objects if they are same or not.
>>> a, b = 1, 1 >>> id(a), id(b) (140188520280384, 140188520280384) >>> tc.assertIs(a, b)
>>> b = 'c' >>> id(a), id(b) (140188520280384, 140188521613808) >>> tc.assertIs(a, b, msg=f'a={a} and b={b} are not the same objects') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3.8/unittest/case.py", line 1193, in assertIs self.fail(self._formatMessage(msg, standardMsg)) File "/usr/lib/python3.8/unittest/case.py", line 753, in fail raise self.failureException(msg) AssertionError: 1 is not 'c' : a=1 and b=c are not the same objects
Similarly, we have methods like assertIsNone, assertIsNotNone, assertIn, assertNotIn, assertIsInstance, assertIsNotInstance.
assertRaises / assertRaisesRegex
Test that an exception is raised when callable is called with any positional or keyword arguments that are also passed to assertRaises(). The test passes if exception is raised, is an error if another exception is raised, or fails if no exception is raised.
>>> tc.assertRaises(ZeroDivisionError, lambda x: x/(x-1), 1) >>> >>> tc.assertRaises(ZeroDivisionError, lambda x: x/(x-1), 2) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3.8/unittest/case.py", line 816, in assertRaises return context.handle('assertRaises', args, kwargs) File "/usr/lib/python3.8/unittest/case.py", line 202, in handle callable_obj(*args, **kwargs) File "/usr/lib/python3.8/unittest/case.py", line 224, in __exit__ self._raiseFailure("{} not raised by {}".format(exc_name, File "/usr/lib/python3.8/unittest/case.py", line 164, in _raiseFailure raise self.test_case.failureException(msg) AssertionError: ZeroDivisionError not raised by <lambda>
If you want to check for the exact error message, you can use the following method,
>>> tc.assertRaisesRegex(ValueError, "invalid literal for.*XYZ'$", int, 'XYZ') >>> >>> tc.assertRaisesRegex(ValueError, "invalid literal for.*XYZ'$", int, 'ABC') ValueError: invalid literal for int() with base 10: 'ABC' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3.8/unittest/case.py", line 1357, in assertRaisesRegex return context.handle('assertRaisesRegex', args, kwargs) File "/usr/lib/python3.8/unittest/case.py", line 202, in handle callable_obj(*args, **kwargs) File "/usr/lib/python3.8/unittest/case.py", line 240, in __exit__ self._raiseFailure('"{}" does not match "{}"'.format( File "/usr/lib/python3.8/unittest/case.py", line 164, in _raiseFailure raise self.test_case.failureException(msg) AssertionError: "invalid literal for.*XYZ'$" does not match "invalid literal for int() with base 10: 'ABC'"
Similarly, we have assertWarns and assertWarnsRegex to handle test validations for warnings.
assertLogs
A context manager to test that at least one message is logged on the logger or one of its children, with at least the given level. The object returned by the context manager is a recording helper which keeps tracks of the matching log messages. It has two attributes: records and output.
with self.assertLogs('foo', level='INFO') as cm: logging.getLogger('foo').info('first message') logging.getLogger('foo.bar').error('second message') self.assertEqual(cm.output, ['INFO:foo:first message', 'ERROR:foo.bar:second message'])
assertAlmostEqual / assertNotAlmostEqual
It is useful for asserting floating point numbers when you need to compare two values approximately. By default the rounding is done upto 7 decimal places.
>>> tc.assertAlmostEqual(0.1234567,0.1234568) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3.8/unittest/case.py", line 966, in assertAlmostEqual raise self.failureException(msg) AssertionError: 0.1234567 != 0.1234568 within 7 places (1.0000000000287557e-07 difference) >>> tc.assertAlmostEqual(0.1234567,0.12345678) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3.8/unittest/case.py", line 966, in assertAlmostEqual raise self.failureException(msg) AssertionError: 0.1234567 != 0.12345678 within 7 places (7.99999999995249e-08 difference) >>> tc.assertAlmostEqual(0.1234568,0.12345678)
Similarly, we have assert methods like, assertGreater, assertGreaterEqual, assertLess, assertLessEqual, assertRegex, assertNotRegex, assertCountEqual.
assertCountEqual tests that sequence first contains the same elements as second, regardless of their order. When they don’t, an error message listing the differences between the sequences will be generated.
New assert methods in 3.1
Python 3.1 introduces new assert methods to compare native object types directly. The name of the methods are self-explanatory.
Method Used to compare assertMultiLineEqual(a, b) strings assertSequenceEqual(a, b) sequences assertListEqual(a, b) lists assertTupleEqual(a, b) tuples assertSetEqual(a, b) sets or frozensets assertDictEqual(a, b) dicts
This by no means is a comprehensive overview of all the available assert methods, but however, serves as a catalogue for the most commonly used test assertions in a python unit test script.
In this article, we will use Next.js to build a static blog framework with the design and structure inspired by Jekyll. I've always been a big fan of how
In this article, we will use Next.js to build a static blog framework with the design and structure inspired by Jekyll. I’ve always been a big fan of how Jekyll makes it easier for beginners to setup a blog and at the same time also provides a great degree of control over every aspect of the blog for the advanced users.
Python 3.9 will release today. This article serves as a preview for the new features that will be introduced in the final release.
Bear in mind at the time of writing, the official specification is still in the draft mode. Although it is unlikely to change in the last moment, it could still happen. Hence, it is worthwhile to revisit this article for future updates.
Dict union operator
Two dictionaries can now be combined with the pipe (|) operator. It also supports augmented assignment (|=) expression.
>>> d = { 'foo':'bar', 'spam':'eggs' } >>> e = { 'hello':'world' } >>> d | e { 'foo':'bar', 'spam':'eggs', 'hello':'world' } >>> e | d { 'hello':'world', 'foo':'bar', 'spam':'eggs' } >>> d |= e >>> d { 'foo':'bar', 'spam':'eggs', 'hello':'world' }
Improved support for built-ins
In type annotations, you can now use built-in collection types like list, dict as generic types.
def greet_all(names: list[str]) -> None: for name in names: print("Hello", name)
New string methods for prefix/suffix
The builtin str class now adds two new methods str.removeprefix( prefix ), str.removesuffix( suffix ) to remove prefixes and suffixes respectively from the string object.
>>> text = 'Mr. John Doe Jr' >>> text.removeprefix( 'Mr. ' ) 'John Doe Jr' >>> text.removesuffix( ' Jr' ) 'Mr. John Doe'
Str replace behavior
Serhiy Storchaka first reported about this bug. In the latest release, "".replace("", s, n) now returns s instead of an empty string for all non-zero n. It is now consistent with "".replace("", s).
The zoneinfo module brings support for the IANA time zone database to the standard library. It adds zoneinfo.ZoneInfo, a concrete datetime.tzinfo implementation backed by the system’s time zone data.
>>> from zoneinfo import ZoneInfo >>> from datetime import datetime, timedelta # Daylight saving time >>> dt = datetime(2020, 10, 31, 12, tzinfo=ZoneInfo("America/Los_Angeles")) >>> dt.tzname() 'PDT' # Standard time >>> dt += timedelta(days=7) >>> print(dt) 2020-11-07 12:00:00-08:00 >>> print(dt.tzname()) PST
Miscellaneous
The isocalendar() of datetime.date and isocalendar() of datetime.datetime methods now returns a namedtuple() instead of a tuple. (Contributed by Dong-hee Na in bpo-24416.)
__import__() now raises ImportError instead of ValueError, which used to occur when a relative import went past its top-level package. (Contributed by Ngalim Siregar in bpo-37444.)
It's not surprising that dark mode is the new fad in the town after Apple introduced the dark mode in MacOS. I decided to add a toggle button to switch to the dark theme on my blog.
Let's establish a few ground rules of engagement:
The default theme of the blog must always be the light theme. For example, if the user's device does not define the prefers-color-scheme CSS rule then we apply the default theme.
We must respect the mode the user's device is currently switched to and override the blog's theme to match that mode. For example, if the user's device is using the dark mode, then we apply the .dark theme to the blog and vice versa.
We must always provide the user with an option to toggle between the light and the dark theme for the blog.
If the user's device is running on dark mode and the user decides to switch to the light theme, then all subsequent pages should use the light theme, and vice versa.
The behavior defined in #4 should reset when the tab or the browser window is closed.
Step 1
Define the light and dark theme for your site in CSS. Apply the .light class by default to the body of the document.
If you look under the hood of this website, you will not find any Media queries. That doesn't mean this website isn't responsive. It means it is practically possible to design modern websites with no media queries.
I'll focus on two basic elements that changes as screen size changes which typically warrants using media queries in CSS.
Font-size
I use the clamp() function to scale the font-size between a minimum and a maximum value as the screen size changes with respect to the viewport. More on CSS clamps here.
body { font-size: clamp(18px, 1.25vmax, 1.5em); }
Layout
The site layout is designed using CSS Grids wherein we can use the repeat() function to specify the column template for the grid. repeat accepts auto-fit as one of the keywords to define how the grid should behave to fill the cells in the grid container.
The Back To Top button has been a holy grail along with the hamburger menu. Historically, jQuery was used to animate the jump from the bottom of the page to the top of the page. Is there a modern day equivalent?
This is the previous implementation,
<a role="button" onclick="window.scrollTo(0,0)" href="javascript:void(0)">Back to Top</a>
Recently I learnedabout the Element.scrollIntoView method that scrolls the elements parent container such that the element is visible to user in the viewport. Optionally, we can also define behavior to scroll smoothly.
Hence, here is my updated version of the famous Back to Top button,
<a role="button" onclick="document.body.scrollIntoView({behavior:'smooth'})" href="javascript:void(0)">Back to Top</a>
CanIUse suggests that all modern browsers support the scrollIntoView API, which is cool too.
I deployed the latest version of my website (v9.0) and was quite keen to run the Lighthouse Audit report. Admittedly this is such a small website that getting scores close to 100% is quite easy and not as daunting as a modern web app.
Scored 100! in pretty much all the sections, except the Performance index. I pretty much have a good grip on what needs to be done to get it as close as to 100. Watch out the permalink at the bottom of this page, which links to the latest audit report from Lighthouse.
Improvements
Replace Google fonts with a Base 64 locally hosted version of the font stack.
Inline critical path CSS by eliminating render blocking resources.
Ultimately reduce the overall requests count.
Find an alternative to Google Analytics.
Update: 'Since I wrote this artice, I have managed to score 100% on this blog and I intent to maintain that.
Its been four years since I last wrote an article on this blog. Even though the domain has been active, the content has been lying dormant for a while. To take a positive spin out of this ongoing pandemic, I promised myself to revive this blog and resurrect the old content for everyone's pleasure.
Albeit, there'll be some articles that still needs formatting based on the new layout, but hold your breath, those changes are coming thick and fast.
Specs
The blog still uses Jekyll under the hood supported by Commento as the comment engine which works great for static sites like this. Styling is still a mess and I wouldn't recommend peeking the CSS file as of yet.
VSCode is still the choice editor even after so many years. Netlify takes care of the Nameservers, Custom domain management, Build & Deployment, CDN & caching and practically every other part of the stack under the sun.
Promise
I hereby make a promise to myself to be more active on this blog, and resume from where I left of by sharing my learning on web development in as much detail as possible.
The Python mock library has largely been a black hole when it comes to efficiently unit testing a Python code. Hopefully, this article will help you understand the essential bits to bump up that test coverage.
Whether it is about mocking an object, class, method or a function, in Python, everything can be more or less decomposed into a handful of similar steps. Throughout the article, we would only make use of the patch decorator that can be imported as,
>>> from mock import patch
In-case you're version of Python doesn't come bundled up with the mock library, then you could install it by running the following command, in you're nearest terminal,
$ pip install mock
Mock the return value of a function
Let's say you want to mock the return value of a function f2 that is being called from the function f1.
Let's say we want to mock the value of a variable x local to a function scope of f1.
>>> def f1(): ... x = 1 ... print( x ) ... >>> f1() 1
As per Alex Martelli, this is impossible to mock as explained in this Stackoverflow answer. Instead use a default argument, as part of the function signature.
>>> def f1(x=None): ... x = x or 1 ... print( x ) ... >>> f1() 1
Now it is easy to write a test function, by simply calling the function with the expected value.
A README is one of the first things people see when they find your open source project. It should be helpful, welcoming, and friendly.
Rowan Manning writes about the importance of writing a friendly README for your project, which is perhaps in my opinion is the most underrated yet extremely important part of maintaining a project
Over the weekend, I pushed out a new update to the Atomize IO app and this post is all about, explaining how those numbers are calculated.
If you are unfamiliar with Atomize IO, then make sure you checkout the web app and run it against your website. I have been massively impressed with the idea behind Atomic CSS and always wanted to build something to find out, how much duplicate CSS declarations live inside the CSS codebase of popular websites and by how much they can benefit by adopting the Atomic CSS methodology.
Atomize IO answers both the questions and represents the results with bunch of numbers. In the next section, I will outline the principle and algorithm developed to calculate these numbers.
Principle
Since, Atomic CSS relies on single purpose classes, it manages to avoid duplicate declarations in the CSS codebase. However, a typical CSS codebase is flooded with duplicate declarations scattered across various classes, much thanks to the semantic classes we are used to writing. This leads to redundancy, bloat and maintenance overhead.
Hence, it becomes imperative to identify all the unique property-value pairs, construct a ruleset containing each pair with a unique class selector to identify the ruleset.
Algorithm
Grab the input URL.
Make a request to the URL and get the HTML in response.
Grab all the CSS files that are linked in the HTML.
Extract only those CSS styles that are used in the HTML.
Run through each ruleset and identify all the unique property-value pairs.
Construct a CSS file with each declaration in a new ruleset accompanied by a unique selector.
Now you can count the no. of declarations, selectors, rulesets found in CSS files generated at Step 4 (Before) and Step 6 (After).
Step #4 is only used to generate numbers for Single Page CSS view. Combined CSS view uses the complete CSS at Step 3.
Caveats
There are few caveats in the above algorithm, that you should keep in mind while interpreting these numbers.
The approach doesn't take into account inline styles.
Media queries are not handled separately. This means div { color: red; } and @media screen { div { color: red; } } will be counted as one.
Each vendor prefixed property will have a separate class. This is not the really case when using Atomic CSS styles.
Styling via JavaScript is not taken into the account.
Currently the selectors are named as .styles+{counter} to guarantee uniqueness. I am almost sure that an Atomic classname will be much shorter than this naming scheme.
As it stands, the approach is not completely perfect but still does a darn good job of showcasing the benefits by simply adopting Atomic CSS methodology. Need proof? check the stats of twitter.com here.
