正直 本分 平常心 把该做的事情做好
第1天 2023-12-21

Technical articles need to read.

Question need to solve it.

第38天 01-27


Also Include a new python API service?

第24天 01-13



header; payload; signature

第23天 01-12

Why Kafka is so quick?

sequential I/O <=> random I/O
100MB/s <=> 100KB/s

(append only log)

zero copy

第21天 01-10

column-based db design vs row-based db design

第20天 01-09

Evolution of Yelp’s Elasticsearch-based Ranking Platform

第20天 01-09

B-trees and B+trees

comparsion between B-trees and B+trees
1. B+ trees, search keys can be repeated but this is not the case for B-trees
2. B+ trees allow satellite data to be stored in leaf nodess only, whereas B-trees store data in both leaf and internel nodes.
3. Deleting data from a B+ tree is easier and less time consuming because we only need to remove data from leaf nodes.
4. Leaf nodes in a B+ tree are linked together making range search operations efficient and quick.

第20天 01-09

MySQL Replication

master => slave

Replication is asynchronous.

Changes on the master aren't logged until commit.

Then slave downloads binary log.

Slave lag happens
Even when using semi-synchronous replication.

Strict Tolerance
Must always query from the master.

Reading and writing data in the same transaction - or even the same statement.

bids in an auction

Time-Based Tolerance

Slave must include recent changes made by any session.
Lag must be less than a specific number of seconds.
chat-room messages.

Event-Based Tolerance
Data must include recent changes made by the same session.

Lax Tolerance
Query may read freely from any slave without checking lag.
Reading data that doesn't change.
Reading old data is okay.
count of "likes"

第19天 01-08

How zero copy works.

4 steps
1. From disk data is copy in kernal read buffer. (Performed by the direct memory access DMA engine) reads file contents from the disk and stores them into a kernal address space buffer.
2. From kernal read buffer it copy to application read buffer
3. From application buffer it will copy to kernal socket buffer and put the data into a kernel address space buffer again.
4. From socket buffer it will copy to network buffer.

1. causes the file contents to be copied into a read buffer by the DMA(Direct memory access) engine.
2. Then the data is copied by the kernel into the kernel buffer assoicated with the output socket.
3. From socket buffer it will copy to network buffer.

1.The transferTo() method causes the file contents to be copied into a kernel buffer by the DMA engine.
2. No data is copied into the socket buffer. Instead, only descriptors with information about the data are appended to the socket buffer. The DMA engine passes data directly from the kernel buffer to the Network buffer, thus elimating the remaining final CPU copy.

第15天 01-04

CAP theorem

Every read receives the most recent write or an error.

Every request receives a (non-error) response, without the guarantee that it contains the most recent write.

Partition tolerance
The system continues to operate despite an arbitrary number of messages being dropped by the network between nodes.

第4天 2023-12-24

HTTP2 VS HTTP1.1 difference





先通过非对称加密 再通过对称加密进行通信

第2天 2023-12-22

What's JWT?

Json Web Token

第2天 2023-12-22

ACID Properties in DBMS

A transaction is a single logical unit of work that accesses and possibly modifies the contents of a database. Transactions access data using read and write operations. In order to maintain consistency in a database, before and after the transaction, certain propertoes are followed.

Atomicity, Consistency, Isolation, Durability
1.The entire transaction takes place at once or doesn't happen at all.
2. The database must be consistent before and after the transaction
3. Isolation Multiple transactions occur independently without interference.
4. Durability: the changes of a successful transaction occurs even if the system failure occurs.

第2天 2023-12-22

Redis persistence

RDB Redis Database

AOF Append Only File

RDB advantages
RDB is verey compact single-file point-in-time representation of your Redis data. RDS files are perfect for backups.
RDB is very good for disaster recovery, being a single compact file that can be transfeerred to far data centers.
RDB maximizes Redis performances since the only work the Redis parent process need to do in order to persist is forking a child that will do all the rest.

RDB disadvantages
RDS is not good if you need to minimize the chance of data loss in case Redis stops working

AOF advantages
Using AOF Redis is much more durable
The AOF log is an append-only log
Redis is able to automatically rewire the AOF in background when it gets too big.

AOF advantage
AOF files arew usually bigger than the equaivalent RDB files for the same dataset.
AOF can be slower than RDB depending on the exact fsync policy.

第2天 2023-12-22


interface Definition Language

Protocol Buffers
language-neural, platform-neutral extensible merchanism for serializing structured data.

Token based authentication with Google

第2天 2023-12-22

rate limiter

fixing window
Reset at the start of each sampling period.
it will alow regular traffic spikes to go through the rate limter.

sliding window
huge processing and memory requirements.

leaky bucket (used by NGINX)
It works by incrementing a counter when each request comes in.

Sliding windows to the rescue
It smoothes the traffic spike issue
Very easy to understand and configure
(Very accurate)
0.003% of requests have been wrongly allowed or rate limited

第1天 2023-12-21

《Twitter snowflake is really cool》

Sign bit 1 bit
Timestamp 41 bit
Machine ID 10 bit
Sequence number 12 bit

Advantage & Disadvantages

· It's 64 bit long, it is half the size of UUIDs.
· Scalable (It can accommodate 1024 machines)
· Highly available (Each machine can generate 4096 unique IDs each millisecond)
· Some of the UUID version do not include a timestamp. Have a sortable advantage.
· Preditable for future IDs.
· The maximum timestamp that can be represented in 41 bits is ~69 years.

funny with engineering and system design

获得 3 次爱心
获得 0 次收藏