memory allocator

In C - dynamic memory allocation is done using malloc, realloc, calloc and free.

In linux kernel:
__get_free_pages() allocates memory from buddy system
-> /proc/buddyinfo
kmalloc() returns memory from slab caches
-> with slab allocation, memory chunks suitable to fit data objects of certain type or size are preallocated.
-> good for 32bytes to 32pages
-> /proc/slabinfo

In linux user space:
The GNU C library (glibc) uses an allocator based on dlmalloc ("Doug Lea's Malloc").
Allocated memory contains an 8 or 16 byte overhead for the size of the chunk and usage flags.

For requests below 256 bytes (a "smallbin" request), a simple two power best fit allocator is used. If there are no free blocks in that bin, a block from the next highest bin is split in two.

For requests of 256 bytes or above but below the mmap threshold, dlmalloc use an in-place bitwise trie algorithm.

For requests above the mmap threshold (a "largebin" request), the memory is always allocated using the mmap system call. The threshold is usually 256 KB.


Reference:
http://en.wikipedia.org/wiki/C_dynamic_memory_allocation#dlmalloc
http://www.win.tue.nl/~aeb/linux/lk/lk-9.html#ss9.3
http://en.wikipedia.org/wiki/Slab_allocation
http://en.wikipedia.org/wiki/Trie#Bitwise_tries

TLB

Why Virtual Memory?
 - to handle shortage of memory
   - more active process than physical memory can hold
 - to handle excess memory
   - 32bit processor could support 64GB of RAM
 - to support multiprogramming
   - memory protection
   - memory sharing

Advantages of paging
 - allocation of memory is easy and cheap
 - no external fragmentation
Disadvantages of paging
 - mapping table overhead
 - internal fragmentation

A TLB is part of the chip’s memory-management unit (MMU), and is simply a hardware cache of popular virtual-to-physical address translations. A typical TLB might have 32, 64, or 128 entries.

The MIPS R4000 supports a 32-bit address space with 4KB pages. The VPN translates to up to a 24-bit PFN, and hence can support systems with up to 64GB of (physical) main memory (2^24 4KB pages).
- global bit (G), globally-shared page
- address space identifier (ASID)
- Coherence (C) bits, how a page is cached by the hardware
- a valid bit, a valid translation present in the entry.
- a page mask, for multiple page sizes

if the number of pages a program accesses in a short period of time exceeds the number of pages that fit into the TLB, the program will generate a large number of TLB misses, and thus run quite a bit more slowly.
- database management system (a DBMS), which have certain data structures that are both large and randomly-accessed.

Reference:
http://pages.cs.wisc.edu/~remzi/OSFEP/vm-tlbs.pdf
http://cs.nyu.edu/~gottlieb/courses/2000-01-fall/arch/lectures/lecture-23.html
http://www.slideshare.net/vitlic/linux-memory

encrypt/decrypt with openssl API -2

The symmetric cipher commands allow data to be encrypted or decrypted using various block and stream ciphers using keys based on passwords.

int AES_set_encrypt_key(const uchar *userKey, const int bits, 
    AES_KEY *key);

void AES_cbc_encrypt(const uchar *in, uchar *out, const ulong length, 
    const AES_KEY *key, uchar *ivec, const int enc);

# ./encrypt password1
cb3d7f690989f7237fd4485a582ac5b2fe98be8ebd6e37c50bb4bdd734a95631
# ./decrypt cb3d7f690989f7237fd4485a582ac5b2fe98be8ebd6e37c50bb4bdd734a95631
password1

download encrypt.c from here
download decrypt.c from here

# echo -n "password1" > plain.txt
# openssl enc -aes-256-cbc -nosalt -k "1234567890abcdef" -iv 0 -in plain.txt -out enc.txt
# hexdump -C enc.txt 
00000000  aa 3b 81 85 64 bf 6d b9  35 de 59 c3 36 41 0e f2  |.;..d.m.5.Y.6A..|
# openssl enc -d -aes-256-cbc -nosalt -k "1234567890abcdef" -iv 0 -in enc.txt
password1

Reference:
http://saju.net.in/blog/?p=36
http://stackoverflow.com/questions/9889492/how-to-do-encryption-using-aes-in-openssl
http://www.128bitstudios.com/2010/05/31/file-encryption-with-openssl/