(置顶)
Technical articles need to read.

Question need to solve it.

如何在一台服务器上部署多个service。

Also Include a new python API service?

对称加密
AES

非对称加密
RSA ECC

structure
header; payload; signature

Why Kafka is so quick?

sequential I/O <=> random I/O
100MB/s <=> 100KB/s

(append only log)

zero copy

column-based db design vs row-based db design

Evolution of Yelp’s Elasticsearch-based Ranking Platform

https://www.youtube.com/watch?v=rv-SCLb2oOQ

B-trees and B+trees

comparsion between B-trees and B+trees
1. B+ trees, search keys can be repeated but this is not the case for B-trees
2. B+ trees allow satellite data to be stored in leaf nodess only, whereas B-trees store data in both leaf and internel nodes.
3. Deleting data from a B+ tree is easier and less time consuming because we only need to remove data from leaf nodes.
4. Leaf nodes in a B+ tree are linked together making range search operations efficient and quick.

MySQL Replication

master => slave

Replication is asynchronous.

Changes on the master aren't logged until commit.

Then slave downloads binary log.

Slave lag happens
Even when using semi-synchronous replication.

Strict Tolerance
Must always query from the master.

Reading and writing data in the same transaction - or even the same statement.
INSERT INTO foo SELECT ...

bids in an auction

Time-Based Tolerance

Slave must include recent changes made by any session.
Lag must be less than a specific number of seconds.
chat-room messages.

Event-Based Tolerance
Data must include recent changes made by the same session.

Lax Tolerance
Query may read freely from any slave without checking lag.
Reading data that doesn't change.
Reading old data is okay.
count of "likes"

How zero copy works.

4 steps
1. From disk data is copy in kernal read buffer. (Performed by the direct memory access DMA engine) reads file contents from the disk and stores them into a kernal address space buffer.
2. From kernal read buffer it copy to application read buffer
3. From application buffer it will copy to kernal socket buffer and put the data into a kernel address space buffer again.
4. From socket buffer it will copy to network buffer.

(Improved)
1. causes the file contents to be copied into a read buffer by the DMA(Direct memory access) engine.
2. Then the data is copied by the kernel into the kernel buffer assoicated with the output socket.
3. From socket buffer it will copy to network buffer.


(Best)
1.The transferTo() method causes the file contents to be copied into a kernel buffer by the DMA engine.
2. No data is copied into the socket buffer. Instead, only descriptors with information about the data are appended to the socket buffer. The DMA engine passes data directly from the kernel buffer to the Network buffer, thus elimating the remaining final CPU copy.

CAP theorem

Consistency
Every read receives the most recent write or an error.

Availability
Every request receives a (non-error) response, without the guarantee that it contains the most recent write.

Partition tolerance
The system continues to operate despite an arbitrary number of messages being dropped by the network between nodes.

HTTP2 VS HTTP1.1 difference

用到的三种算法

非对称加密
RSA

对称算法
AES

哈希
SHA-256

先通过非对称加密 再通过对称加密进行通信

What's JWT?

Json Web Token

ACID Properties in DBMS

A transaction is a single logical unit of work that accesses and possibly modifies the contents of a database. Transactions access data using read and write operations. In order to maintain consistency in a database, before and after the transaction, certain propertoes are followed.

ACID
Atomicity, Consistency, Isolation, Durability
1.The entire transaction takes place at once or doesn't happen at all.
2. The database must be consistent before and after the transaction
3. Isolation Multiple transactions occur independently without interference.
4. Durability: the changes of a successful transaction occurs even if the system failure occurs.

Redis persistence

RDB Redis Database

AOF Append Only File

RDB advantages
RDB is verey compact single-file point-in-time representation of your Redis data. RDS files are perfect for backups.
RDB is very good for disaster recovery, being a single compact file that can be transfeerred to far data centers.
RDB maximizes Redis performances since the only work the Redis parent process need to do in order to persist is forking a child that will do all the rest.

RDB disadvantages
RDS is not good if you need to minimize the chance of data loss in case Redis stops working

AOF advantages
Using AOF Redis is much more durable
The AOF log is an append-only log
Redis is able to automatically rewire the AOF in background when it gets too big.

AOF advantage
AOF files arew usually bigger than the equaivalent RDB files for the same dataset.
AOF can be slower than RDB depending on the exact fsync policy.

gRPC

IDL
interface Definition Language

Protocol Buffers
language-neural, platform-neutral extensible merchanism for serializing structured data.

SSL/TLS
Token based authentication with Google

rate limiter

fixing window
Reset at the start of each sampling period.
it will alow regular traffic spikes to go through the rate limter.

sliding window
huge processing and memory requirements.

leaky bucket (used by NGINX)
It works by incrementing a counter when each request comes in.

Sliding windows to the rescue
It smoothes the traffic spike issue
Very easy to understand and configure
(Very accurate)
0.003% of requests have been wrongly allowed or rate limited

《Twitter snowflake is really cool》

Sign bit 1 bit
Timestamp 41 bit
Machine ID 10 bit
Sequence number 12 bit

Advantage & Disadvantages

· It's 64 bit long, it is half the size of UUIDs.
· Scalable (It can accommodate 1024 machines)
· Highly available (Each machine can generate 4096 unique IDs each millisecond)
· Some of the UUID version do not include a timestamp. Have a sortable advantage.
· Preditable for future IDs.
· The maximum timestamp that can be represented in 41 bits is ~69 years.