Profile

Cover photo
Yann Collet
Works at Orange
Attended L'Ecole Nationale Supérieure de Télécommunications de Bretagne
71 followers|379,888 views
AboutPostsPhotosVideos+1's

Stream

Yann Collet

Shared publicly  - 
1
Add a comment...

Yann Collet

Shared publicly  - 
 
Sampling, or a faster LZ4
 Quite some time ago, I've been experimenting with some unusual sampling methods, in an attempt to find a better way to compress data with LZ4. The main idea was as follows : LZ4 hash table is getting filled pretty quickly, due to its small size. It becomes...
 Quite some time ago, I've been experimenting with some unusual sampling methods, in an attempt to find a better way to compress data with LZ4. The main idea was as follows : LZ4 hash table is getting filled pretty quickly, d...
4
AJ Baxter's profile photoYann Collet's profile photo
4 comments
 
In which case, you may also appreciate a new function within next release, r129.

It's called LZ4_compress_destSize() and reverse the logic : it tries to fit as much input as possible into a max-sized destination buffer.
Add a comment...

Yann Collet

Shared publicly  - 
 
 ZSTD, short for Z-Standard, is a new lossless compression algorithm, aiming at providing both great compression ratio and speed for your standard compression needs. "Standard" translates into everyday situations which neithe...
2
Add a comment...

Yann Collet

Shared publicly  - 
 
Portability woes : Endianess and Alignment (Part 2)
In the  previous part , endianess and 32/64 bits detection were detailed. Now is the time to have a look at more complex memory alignment troubles. Memory Access Alignment  is a lesser known issue, but its impact can be huge : it will crash your program or ...
1
Add a comment...

Yann Collet

Shared publicly  - 
 
Counting bytes fast - little trick from FSE
 An apparently trivial and uninteresting task got nonetheless receive some special optimization care within FSE : counting the bytes (or 2-bytes shorts when using the U16 variant ). It seems a trivial task, and could indeed be realized by a single-lie funct...
 An apparently trivial and uninteresting task got nonetheless receive some special optimization care within FSE : counting the bytes (or 2-bytes shorts when using the U16 variant). It seems a trivial task, and could indeed be...
1
AJ Baxter's profile photoYann Collet's profile photo
2 comments
 
Good artists copy, great artists steal !
Add a comment...

Yann Collet

Shared publicly  - 
 
xxHash wider : 64-bits
 The initial version of xxHash was created in a bid to find a companion error detection algorithm for LZ4 decoder. The objective was set after discovering that usual implementation of CRC32 were so slow that the overall process of decoding + error check wou...
1
Add a comment...
Have him in circles
71 people
Alexander Melnikov's profile photo
Stéphane Cocquereaumont's profile photo
Sami Yurdakul's profile photo
Francesc Alted's profile photo
Greg Leclercq's profile photo
Arkadiusz S's profile photo
Piotr Tarsa's profile photo
Adrien Grand's profile photo
Federico Oro Vojacek's profile photo

Yann Collet

Shared publicly  - 
 
Changing course
 Since I started programming a few years ago, and selected data compression as my little hobby and obsession, I nonetheless remained a part-time, amateur, programmer. My then real-life job was Telecommunication Project Manager, later morphed into Marketing ...
2
Takayuki Matsuoka's profile photoYann Collet's profile photo
2 comments
 
Thanks, you're welcomed Takayuki.

My communication possibilities will be a bit limited in the next few days, but hopefully, I'll be back online by next week. Rgds
Add a comment...

Yann Collet

Shared publicly  - 
 
The Bloom filter is a space efficient, probabilistic data structure, designed to test the membership of elements to a set. The trade-off for being a space efficient data structure is it may return false positives, but always returns definite negatives: Meaning Bloom filters can accurately test ...
1
Add a comment...

Yann Collet

Shared publicly  - 
 
ZSTD - A stronger compression algorithm
  ZSTD , short for Z-Standard, is a new lossless compression algorithm, aiming at providing both great compression ratio and speed for your standard compression needs. "Standard" translates into everyday situations which neither look for highest possible ra...
 ZSTD, short for Z-Standard, is a new lossless compression algorithm, aiming at providing both great compression ratio and speed for your standard compression needs. "Standard" translates into everyday situations which neithe...
7
Yann Collet's profile photoAJ Baxter's profile photo
14 comments
 
i wrote a set of batch files too, instead of using make.   It got the job done!     
Add a comment...

Yann Collet

Shared publicly  - 
 
Portability woes : Endianess and Alignment (Part 1)
 In creating an ultra-portable code, able to be compiled on (almost) every platform, there are some unusual problems to take into consideration. Listing them by order of increasing complexity, this post will review in detail address space, endianess and ali...
 In creating an ultra-portable code, able to be compiled on (almost) every platform, there are some unusual problems to take into consideration. Listing them by order of increasing complexity, this post will review in detail ...
1
Add a comment...

Yann Collet

Shared publicly  - 
 
Pointer overflow vs RCE : an analysis of LZ4 literal copy attack
 Last week, when a blog announced to the wild that it was possible to overflow a pointer within LZ4, I immediately produced a fix within the next few hours to protect users, without checking how the code would naturally behave in such circumstance. After al...
1
Add a comment...
People
Have him in circles
71 people
Alexander Melnikov's profile photo
Stéphane Cocquereaumont's profile photo
Sami Yurdakul's profile photo
Francesc Alted's profile photo
Greg Leclercq's profile photo
Arkadiusz S's profile photo
Piotr Tarsa's profile photo
Adrien Grand's profile photo
Federico Oro Vojacek's profile photo
Education
  • L'Ecole Nationale Supérieure de Télécommunications de Bretagne
    Télécommunications, 1993 - 1996
Links
Work
Employment
  • Orange
    Marketing M2M, 2009 - present
Basic Information
Gender
Male
Yann Collet's +1's are the things they like, agree with, or want to recommend.
Strict Aliasing
dbp-consulting.com

So What's The Problem? There's a lot of confusion about strict aliasing rules. The main source of people's confusion is that there are two d

Bloom Filters - Explained ~ Tech-Effigy
techeffigytutorials.blogspot.com

The Bloom filter is a space efficient, probabilistic data structure, designed to test the membership of elements to a set. The trade-off for

ZSTD - A stronger compression algorithm
fastcompression.blogspot.com

ZSTD, short for Z-Standard, is a new lossless compression algorithm, aiming at providing both great compression ratio and speed for your sta

Performance of various general compression algorithms - some of them are...
java-performance.info

This article compares several Java general compression algorithm implementations - you can compress your data at over 300 Mb/sec using some

Wifi Analyzer
market.android.com

Turns your android phone into a Wi-Fi analyzer!! Shows the Wi-Fi channels around you. Helps you to find a less crowded channel for your wire

bourse aux jouets - Hauts-de-Seine - 92
bourse-aux-jouets.org

découvrez les dates des bourses aux jouets dans les Hauts-de-Seine - 92

Counting bytes fast - little trick from FSE
fastcompression.blogspot.com

An apparently trivial and uninteresting task nonetheless received some special optimization care within FSE : counting the bytes (or 2-bytes

Let's move on
fastcompression.blogspot.com

I've received an answer from Don Bailey. He blames the situation on a lack of communication. OK. In an attempt to bring the discussion to a

Streaming API for LZ4
fastcompression.blogspot.com

For quite some time, the LZ4 Streaming API project has been started and delayed, as other priorities stepped in the way. To be fair, one imp

Taking advantage of unequalities to provide better compression
fastcompression.blogspot.com

When starting investigation on ANS properties, in November 2013, I stumbled upon the fact that positions in the table are not equivalent. Ba

Ultra-fast normalization
fastcompression.blogspot.com

Today's objective is to use the lessons learned when defining the perfect normalization algorithm to design a new, better, fast normalizatio

Perfect Normalization
fastcompression.blogspot.com

People keeping an eye on the github repository of FSE may have noticed the apparition a new function, called FSE_normalizeCountHC(). Let's d

Better normalization, for better compression
fastcompression.blogspot.com

A secondary issue to deal with when implementing an FSE entropy coder is the requirement to normalize statistics. Basically, it consists in

FSE tricks - Memory efficient subrange maps
fastcompression.blogspot.com

With last blog post, we were left with a description of the FSE compression algorithm. However, the code associated was a bit cryptic. So le

FSE encoding : how it works
fastcompression.blogspot.com

Quite unusually, we have started this serie on FSE by focusing on the decompression part. It may seem strange to try to decode a data we hav

FSE decoding : wrap up
fastcompression.blogspot.com

With all the previous articles written on FSE, it seems we now have all elements to build a complete FSE decoder. Let's wrap up all this lea

FSE : distributing symbol values
fastcompression.blogspot.com

Note : Charles Bloom develops around the same issue on his blog : Understanding ANS - 8. Following the last article on defining optimal sub-

FSE : Defining optimal subranges
fastcompression.blogspot.com

Note : Charles Bloom started an independent in-depth analysis of ANS on his blog, which is definitely worth the read. As stated in earlier F

A comparison of Arithmetic Encoding with FSE
fastcompression.blogspot.com

Arithmetic encoding was invented in the 70's by Jorma Rissanen. After a few decades during which the technique almost stagnated, partly as a