Topics 14
Last updated
Last updated
The diagram below shows the differences. Both approaches are very secure, but the implementations are different. To understand the difference, we break down the process into two flows.
1 Registering your credit card flow
2 Basic payment flow
The registration flow is represented by steps 1~3 for both cases. The difference is: Apple Pay: Apple doesn’t store any card info. It passes the card info to the bank. Bank returns a token called DAN (device account number) to the iPhone. iPhone then stores DAN into a special hardware chip. Google Pay: When you register the credit card with Google Pay, the card info is stored in the Google server. Google returns a payment token to the phone.
When you click the “Pay” button on your phone, the basic payment flow starts. Here are the differences: Apple Pay: For iPhone, the e-commerce server passes the DAN to the bank. Google Pay: In the Google Pay case, the e-commerce server passes the payment token to the Google server. Google server looks up the credit card info and passes it to the bank.
In the diagram, the red arrow means the credit card info is available on the public network, although it is encrypted.
The diagram below shows the differences between a forward proxy and a reverse proxy.
A forward proxy is a server that sits between user devices and the internet.
A forward proxy is commonly used for:
Protect clients
Avoid browsing restrictions
Block access to certain content
A reverse proxy is a server that accepts a request from the client, forwards the request to web servers, and returns the results to the client as if the proxy server had processed the request.
A reverse proxy is good for:
Protect servers
Load balancing
Cache static contents
Encrypt and decrypt SSL communications
The CAP theorem is one of the most famous terms in computer science, but I bet different developers have different understandings. Let’s examine what it is and why it can be confusing.
CAP theorem states that a distributed system can't provide more than two of these three guarantees simultaneously.
Consistency: consistency means all clients see the same data at the same time no matter which node they connect to.
Availability: availability means any client which requests data gets a response even if some of the nodes are down.
Partition Tolerance: a partition indicates a communication break between two nodes. Partition tolerance means the system continues to operate despite network partitions.
The “2 of 3” formulation can be useful, but this simplification could be misleading.
Picking a database is not easy. Justifying our choice purely based on the CAP theorem is not enough. For example, companies don't choose Cassandra for chat applications simply because it is an AP system. There is a list of good characteristics that make Cassandra a desirable option for storing chat messages. We need to dig deeper.
“CAP prohibits only a tiny part of the design space: perfect availability and consistency in the presence of partitions, which are rare”. Quoted from the paper: CAP Twelve Years Later: How the “Rules” Have Changed.
The theorem is about 100% availability and consistency. A more realistic discussion would be the trade-offs between latency and consistency when there is no network partition. See PACELC theorem for more details.
Is the CAP theorem really useful? I think it is still useful as it opens our minds to a set of tradeoff discussions, but it is only part of the story. We need to dig deeper when picking the right database.
What is CDN (Content Delivery Network)? How does CDN make content delivery faster?
The diagram below shows why.
A CDN refers to geographically distributed servers (also called edge servers) that provide fast delivery of static and dynamic content.
With CDN, users don’t need to retrieve content (music, video, files, pictures, etc.) from the origin server. Instead, the content is cached at CDN nodes around the globe, and users can retrieve the content from nearby CDN nodes.
The benefits of CDN are:
Reducing latency
Reducing bandwidth
Improving website security, especially protecting against DDoS (Distributed Denial-of-Service) attack
Increasing content availability
If your answer is on-premise servers and monolith, you would likely fail the interview, but that's how it is built in reality!
What people think it should look like The interviewer is probably expecting something on the left side.
Microservice is used to decompose the system into small components.
Each service has its own database. Use cache heavily.
The service is sharded.
The services talk to each other asynchronously through message queues.
The service is implemented using Event Sourcing with CQRS.
Showing off knowledge in distributed systems such as eventual consistency, CAP theorem, etc.
What it actually is Stack Overflow serves all the traffic with only 9 on-premise web servers, and it’s on monolith! It has its own servers and does not run on the cloud.
This is contrary to all our popular beliefs these days.
One picture is worth a thousand words.
iQIYI is one of the largest online video sites in the world, with over 500 million monthly active users. Let's look at how they choose relational and NoSQL databases.
The following databases are used at iQIYI:
MySQL
Redis
TiDB: a hybrid transactional/analytical processing (HTAP) distributed database
Couchbase: distributed multi-model NoSQL document-oriented database
TokuDB: open-source storage engine for MySQL and MariaDB.
Big data analytical systems, like Hive and Impala
Other databases, like MongoDB, HiGraph, and TiKV
The database selection trees below explain how they choose a database.
The diagram below shows how data is stored in column-based DB.
When to use
The table is a wide table with many columns.
The queries and calculations are on a small number of columns.
A lot of the columns contain a few distinct values.
Benefits of column-based DB
Higher data compression rates.
Higher performance on OLAP functions.
No need for additional indexes
How do fintech startups find new opportunities among so many payment companies? What do PayPal, Stripe, and Square do exactly?
Steps 0-1: The cardholder opens an account in the issuing bank and gets the debit/credit card. The merchant registers with ISO (Independent Sales Organization) or MSP (Member Service Provider) for in-store sales. ISO/MSP partners with payment processors to open merchant accounts.
Steps 2-5: The acquiring process. The payment gateway accepts the purchase transaction and collects payment information. It is then sent to a payment processor, which uses customer information to collect payments. The acquiring processor sends the transaction to the card network. It also owns and operates the merchant’s account during settlement, which doesn’t happen in real-time.
Steps 6-8: The issuing process. The issuing processor talks to the card network on the issuing bank’s behalf. It validates and operates the customer’s account.
I’ve listed some companies in different verticals in the diagram. Notice payment companies usually start from one vertical, but later expand to multiple verticals.
The diagram below shows Web 1.0/Web 2.0/Web 3.0 from a bird's-eye view.
Web 1.0 - Read Only Between 1991 and 2004, the internet is like a catalog of static pages. We can browse the content by jumping from one hyperlink to another. It doesn’t provide any interactions with the content.
Web 2.0 - Read Write From 2004 to now, the internet has evolved to have search engines, social media apps, and recommendation algorithms backed apps.
Because the apps digitalize human behaviors and persist user data when users interact with these apps, big companies leverage user data for advertisements, which becomes one of the main business models in Web 2.0.
That’s why people say the apps know you better than your friends, family, or even yourself.
Web 3.0 - Read Write Own The idea has been discussed a lot recently due to the development of blockchain and decentralized apps. The creators’ content is stored on IPFS (InterPlanetary File System) and owned by the users. If apps want to access the data, they need to get authorization from the users and pay for it.
In Web 3.0, the ownership change may lead to some major innovations.
It is a great example of why a simple feature may take much longer to develop than many people think.
When we have a great design, users may not notice the complexity because it feels like the feature just working as intended.
Scale from zero to millions of users, design a news feed system, design YouTube, and many more. The text-based course is designed to make you a better software engineer and ace the system design interview.
Note: This course contains the same content as my best-selling system design interview books (volume 1 and volume 2). If you already own those two books, there is no need to buy the course.
A common practice is to use RUDP (Reliable UDP). It adds a reliable mechanism on top of UDP to provide much lower latency than TCP and guarantee accuracy.
The diagram below shows how reliable data delivery is implemented in online gaming to get eventually-synchronized states.
Suppose there is a big fight in a simulation shooter game. Characters A, B, and C open fires in sequence. How does the game server transmit the states from the game server to the game client?
Steps 1 and 2 - Character A opens fires. The packet (packet 0) is sent to the client. The client acknowledges the server.
Step 3 - Character B opens fire. The packet is lost during transmission.
Steps 4 and 5 - Character C opens fire. The packet (packet 2) is sent to the client. Since the last successfully delivered packet is packet 0, the client knows that packet 1 is lost, so packet 2 is buffered on the client side. The client acknowledges the server for the reception of packet 2.
Steps 6 and 7 - The server doesn’t receive the ack for packet 1 for a while, so it resends packet 1. When the client receives packet 1, all the subsequent packets become effective, so packets 1 and 2 become “delivered”. The client then acknowledges the server for the reception of packet 1. No packets are buffered at this point.
