Topics 9
Last updated
Last updated
The diagram below illustrates how different types of joins work.
(INNER) JOIN: returns only matching rows between both tables.
LEFT (OUTER) JOIN: returns all matching rows, and non-matching rows from the left table.
RIGHT (OUTER) JOIN: returns all the matching rows, and non-matching rows from the right table.
FULL (OUTER) JOIN: returns all the rows from both left and right tables, including non-matching rows.
How do we properly deal with HTTP errors on the browser side? And how do you handle them correctly on the server side when the client side is at fault?
From the browser's point of view, the easiest thing to do is to try again and hope the error just goes away. This is a good idea in a distributed network, but we also have to be very careful not to make things worse. Here’s two general rules:
For 4XX http error code, do not retry.
For 5XX http error code, try again carefully.
So which things should we do carefully in the browser? We definitely should not overwhelm the server with retried requests. An algorithm named exponential backoff might be able to help. It controls two things:
The latency between two retries. The latency will increase exponentially.
The number of retries is usually capped.
Will all browsers handle their retry logic in a graceful way? Most likely not. So the server has to take care of its own safety. A common way to control the flow of HTTP requests is to set up a flow control gateway in front of the server. This provides two useful tools:
Rate limiter, which limits how often a request can be made. It has two slightly different choices; the token bucket and the leaky bucket.
Circuit breaker. This will stop the HTTP flow immediately when the error threshold is exceeded. After a set amount of time, it will only let a limited amount of HTTP traffic through. If everything works well, it will slowly let all HTTP traffic through.
We should be able to handle intermittent errors effectively with exponential backoff in the browser and with a flow control gateway on the server side. Any remaining issues are real errors that need to be fixed carefully.
Over to you: Both token bucket and leaky bucket can be used for rate limiting. How do you know which one to pick?
If you prefer reading, you can read it here:
Hypertext Transfer Protocol Secure (HTTPS) is an extension of the Hypertext Transfer Protocol (HTTP.) HTTPS transmits encrypted data using Transport Layer Security (TLS.) If the data is hijacked online, all the hijacker gets is binary code.
How is the data encrypted and decrypted?
Step 1 - The client (browser) and the server establish a TCP connection.
Step 2 - The client sends a “client hello” to the server. The message contains a set of necessary encryption algorithms (cipher suites) and the latest TLS version it can support. The server responds with a “server hello” so the browser knows whether it can support the algorithms and TLS version.
The server then sends the SSL certificate to the client. The certificate contains the public key, hostname, expiry dates, etc. The client validates the certificate.
Step 3 - After validating the SSL certificate, the client generates a session key and encrypts it using the public key. The server receives the encrypted session key and decrypts it with the private key.
Step 4 - Now that both the client and the server hold the same session key (symmetric encryption), the encrypted data is transmitted in a secure bi-directional channel.
Why does HTTPS switch to symmetric encryption during data transmission? There are two main reasons:
Security: The asymmetric encryption goes only one way. This means that if the server tries to send the encrypted data back to the client, anyone can decrypt the data using the public key.
Server resources: The asymmetric encryption adds quite a lot of mathematical overhead. It is not suitable for data transmissions in long sessions.
Would it be nice if the code we wrote automatically turned into architecture diagrams?
I recently discovered a Github repo that does exactly this: Diagram as Code for prototyping cloud system architectures.
𝐖𝐡𝐚𝐭 𝐝𝐨𝐞𝐬 𝐢𝐭 𝐝𝐨?
Draw the cloud system architecture in Python code.
Diagrams can also be rendered directly inside the Jupyter Notebooks.
No design tools are needed.
Supports the following providers: AWS, Azure, GCP, Kubernetes, Alibaba Cloud, Oracle Cloud, etc.
What does Availability mean when you design a system?
In the famous CAP theorem by computer scientist Eric Brewer, Availability means all (non-failing) nodes are available for queries in a distributed system. When you send out requests to the nodes, a non-failing node will return a reasonable response within a reasonable amount of time (with no error or timeout).
Usually, we design a system for high availability. For example, when we say the design target is 4-9’s, it means the services should be up 99.99% of the time. This also means the services can only be down for 52.5 minutes per year.
Note that availability only guarantees that we will receive a response; it doesn’t guarantee the data is the most up-to-date.
The diagram below shows how we can turn a single-node “Product Inventory” into a double-node architecture with high availability.
🔹Primary-Backup: the backup node is just a stand-by, and the data is replicated from primary to backup. When the primary fails, we need to manually switch to the backup node. The backup node might be a waste of hardware resources.
🔹Primary-Secondary: this architecture looks similar to primary-backup architecture, but the secondary node can take read requests to balance the reading load. Due to latency when replicating data from primary to secondary, the data read from the secondary may be inconsistent with the primary.
🔹Primary-Primary: both nodes act as primary nodes, both nodes can handle read/write operations, and the data is replicated between the two nodes. This type of architecture increases the throughput, but it has limited use cases. For example, if both nodes need to update the same product, the final state might be unpredictable. Use this architecture with caution!
If we deploy the node on Amazon EC2, which has 90% availability, the double-node architecture will increase availability from 90% to 99%.
Over to you: We’ve covered availability, but do these 3 architecture types also guarantee consistency, or not? Let us know your thoughts!
Do you know how an email is delivered?
When I first learned how similar email is to traditional ‘snail’ mail, I was surprised. Maybe you will be, too. Allow me to explain.
In the physical world, if I want to send a postcard to a friend, I would put it in a nearby mailbox. The postal office collects the mail regularly and relays it to the destination postal office. This postal office then puts the postcard in my friend’s mailbox. This process usually takes a few days and my friend receives my gratitude in paper form.
Email functions in a similar way. The terminology changes because it is an internet-based solution, but the fundamentals are the same:
Instead of putting mail in a mailbox, the sender pushes an email to the Sender Mail Server using MUA (mail user agent,) such as Outlook or Gmail.
Instead of using postal offices to relay mail, MTA (mail transmission agent) relays the email. It communicates via the SMTP protocol.
The email is received by the Receiver Mail Server. It stores the email to the Mailbox by using MDA (mail delivery agent.) The receiver uses MUA to retrieve the email using the POP3/IMAP protocol.
Over to you: if a recipient's email address is incorrect, the email will be returned to you. Do you know how does that work?
The mind map below shows an extensive outline of payment knowledge.
This gives us 𝐚 𝐭𝐨𝐩-𝐝𝐨𝐰𝐧 𝐯𝐢𝐞𝐰 𝐨𝐟 𝐭𝐡𝐞 𝐢𝐧𝐝𝐮𝐬𝐭𝐫𝐲 and an overview of how the payment systems work.
Notice that different countries have different payment frameworks. But in general, the payment industry is composed of below parts:
🔹 Regulatory Authority 🔹 Central Bank 🔹 Commercial Banks 🔹 Non-Bank Payment Companies 🔹 Payment Systems 🔹 Clearing Networks 🔹 Settlement Systems
▶️ 𝐌𝐨𝐫𝐞 𝐔𝐬𝐞𝐟𝐮𝐥 𝐌𝐚𝐭𝐞𝐫𝐢𝐚𝐥𝐬 for your reference:
Payment Systems in the U.S. by GlenBrook
BIS (Bank for International Settlements) Website
PayPal documents
Stripe documents
SWIFT documents
Which protocol does Zoom use for video streaming, TCP or UDP?
Let’s review the differences first.
🔹 The primary difference between TCP and UDP is that TCP is connection-based whereas UDP is connectionless.
Connection-based: It implies that all messages will arrive and arrive in the correct order.
Connectionless: It does not guarantee order or completeness.
🔹 The second difference is that UDP is faster than TCP.
UDP does not require ACK message back
UDP has no flow control
No duplication verification at the receiving end
Shorter header
UDP sacrifices correctness for speed. Users generally prefer smooth video calls and UDP is indeed the default choice for Zoom.
Over to you: the HTTP protocol used to be based on TCP, but the latest protocol HTTP/3 is based on UDP. Do you know why?
What is Distributed SQL? Why do we need it?
Google's Spanner popularized the term “distributed SQL” database in 2012. Distributed SQL databases automatically replicate data across multiple nodes and are strongly consistent. Paxos or Raft algorithms are commonly used to achieve consensus across multiple nodes.
Examples of Distributed SQL databases: Google Spanner, Amazon Aurora, CockroachDB, YugabyteDB, TiDB, etc.
A few weeks ago, I met the CTO of TiDB ED Huang and a few other team members. We discussed how Distributed SQL databases evolved and how TiDB developed its own open-sourced database. After that conversation, I read more documentation/source code about it and I found it to be an interesting case study. I want to share my learning here.
Terms:
OLTP = Online transactional processing
OLAP = Online analytical processing
HTAP = Hybrid transaction/analytical processing
🔹The life of an OLTP query (marked with blue sequence numbers):
Step 1. A client sends a MySQL query and the query is interpreted by TiDB, a stateless SQL layer that interprets the MySQL protocol.
Step 2: TiDB requests data mapping/placement information from the placement driver (PD).
Step 3: PD responds with data mapping/ placement instructions & timestamp.
Step 4: TiDB executes queries on TiKV servers (row-based storage).
Step 5, 6: Query results are sent back to the client.
🔹The life of a complex OLAP query: marked with yellow sequence numbers.
Over to you: do you think the terms OLTP and OLAP have become obsolete or are they still very relevant? When should we use distributed SQL databases vs traditional relational databases?
How do we implement a non-blocking queue? What are the differences between blocking and non-blocking algorithms?
The terms we use when discussing blocking and non-blocking algorithms can be confusing, so let’s start by reviewing the terminology in the concurrency area with a diagram.
🔹blocking
The blocking algorithm uses locks. Thread A acquires the lock first, and Thread B might wait for arbitrary lengthy periods if Thread A gets suspended while holding the lock. This algorithm may cause Thread B to starve.
🔹non-blocking:
The non-blocking algorithm allows Thread A to access the queue, but Thread A must complete a task in a certain number of steps. Other threads like Thread B may still starve due to the rejections.
This is the main difference between blocking and non-blocking algorithms: The blocking algorithm blocks Thread B until the lock is released. A non-blocking algorithm notifies Thread B that access is rejected.
🔹starvation-free:
Thread Starvation means a thread cannot acquire access to certain shared resources and cannot proceed. Starvation-free means this situation does not occur.
🔹wait-free:
All threads can complete the tasks within a finite number of steps.
𝘞𝘢𝘪𝘵-𝘧𝘳𝘦𝘦 = 𝘕𝘰𝘯-𝘉𝘭𝘰𝘤𝘬𝘪𝘯𝘨 + 𝘚𝘵𝘢𝘳𝘷𝘢𝘵𝘪𝘰𝘯-𝘧𝘳𝘦𝘦
➡️ Non-Blocking Queue implementation
We can use Compare and Swap (CAS) to implement a non-blocking queue. The diagram below illustrates the algorithm.
➡️ Benefits
No thread suspension. Thread B can get a response immediately and then decide what to do next. In this way, the thread latency is greatly reduced.
No deadlocks. Threads A and B do not wait for the lock to release, meaning that there is no possibility of a deadlock occurring.
