David Johnston's interests

David Johnston's posts

For those that might care, I have posted software that implements the NIST SP800-22 Rev1a suite of random number tests. Arguably it sucks less than the NIST software which crashes a lot.

https://github.com/dj-on-github/sp800_22_tests

https://github.com/dj-on-github/sp800_22_tests

'ent' (http://www.fourmilab.ch/random/) has been a popular tool for measuring statistics of unprocessed random data from entropy sources. However it has some limitations. The CSV output is not very useful since it can only handle one file, the SCC is lag-1 only and treats all data as cyclical (which it never is), the symbol size is limited to 1 or 8 bits and it doesn't handle hex data well.

I've written a replacement 'djent' that better meets my needs. If it's of interest to you, try it out. It's new so there may be bugs remaining.

https://github.com/dj-on-github/djent

# djent -h

Usage: djent [-b] [-l <n>] [-c] [-u] [-h] [-f] [-t] [filename] [filename2] ...

Compute statistics of random data.

Author: David Johnston, dj@deadhat.com

-l <n> --symbol_length=<n> Treat incoming data symbols as bitlength n. Default is 8.

-b --binary Treat incoming data as binary. Default bit length will be -l 1

-c --occurrence Print symbol occurrence counts

-w --scc_wrap Treat data as cyclical in SCC

-n <n> --lagn=<n> Lag gap in SCC. Default=1

-f --fold Fold uppercase letters to lower case

-t --terse Terse output

-h or -u --help Print this text

It handles two formats - Hex (the default) and Binary. In hex mode, the hex is converted to binary on the fly before being tested. It strips non hex characters and 0x prefixes. The symbol size is independent of the representation. It defaults to 8 for hex mode and 1 for binary mode but can be set to be be from 1 to 32 bits. Larger symbol sizes are limited by the resulting 2^n occurrence count table.

E.G.

djent hexfile.hex

opening hexfile.hex as hex text

Symbol Size(bits) = 8

Shannon IID Entropy = 7.983000 bits per symbol

Optimal compression would compress by 0.212501 percent

Chi square: symbol count=10240, distribution=238.00, randomly exceeds 77.05 percent of the time

Mean = 127.863965

Monte Carlo value for Pi is 3.139508 (error 0.07 percent).

Serial Correlation = -0.014567

If you want to measure the properties of ascii data (not hex), then use binary mode with 8 bit symbols.

# djent -b -l 8 hexfile.hex

opening hexfile.hex as binary

Symbol Size(bits) = 8

Shannon IID Entropy = 4.052783 bits per symbol

Optimal compression would compress by 49.340206 percent

Chi square: symbol count=20800, distribution=303260.11, randomly exceeds 0.00 percent of the time

Mean = 57.392740

Monte Carlo value for Pi is 4.000000 (error 27.32 percent).

Serial Correlation = -0.008395

Multiple files can be fed in. Terse mode (-t) is useful to format the output from multiple files.

#djrandom -b -k 10 -m correlated --correlation=0.2 > corr0p2.bin

#djrandom -b -k 10 -m correlated --correlation=0.3 > corr0p3.bin

#djrandom -b -k 10 -m correlated --correlation=0.4 > corr0p4.bin

#djrandom -b -k 10 -m correlated --correlation=0.5 > corr0p5.bin

# djent -b -t corr0p*.bin

0, File-bytes, Entropy, Chi-square, Mean, Monte-Carlo-Pi, Serial-Correlation, Filename

1, 10240, 1.000000, 88.057350, 0.499738, 2.933177, 0.201680, corr0p2.bin

2, 10240, 0.999938, 0.808874, 0.504626, 2.769050, 0.300127, corr0p3.bin

3, 10240, 0.999975, 9.083176, 0.497046, 2.780774, 0.399825, corr0p4.bin

4, 10240, 0.999998, 66.992535, 0.499255, 2.698710, 0.500628, corr0p5.bin

3 bit symbols? Not a problem. Use the -l <n> directive.

# djent -b -l 3 binfile.bin

opening binfile.bin as binary

Symbol Size(bits) = 3

Shannon IID Entropy = 2.999628 bits per symbol

Optimal compression would compress by 0.012417 percent

Chi square: symbol count=27304, distribution=14.18, randomly exceeds 4.81 percent of the time

Mean = 3.475974

Monte Carlo value for Pi is 3.139508 (error 0.07 percent).

Serial Correlation = 0.001389

I've written a replacement 'djent' that better meets my needs. If it's of interest to you, try it out. It's new so there may be bugs remaining.

https://github.com/dj-on-github/djent

# djent -h

Usage: djent [-b] [-l <n>] [-c] [-u] [-h] [-f] [-t] [filename] [filename2] ...

Compute statistics of random data.

Author: David Johnston, dj@deadhat.com

-l <n> --symbol_length=<n> Treat incoming data symbols as bitlength n. Default is 8.

-b --binary Treat incoming data as binary. Default bit length will be -l 1

-c --occurrence Print symbol occurrence counts

-w --scc_wrap Treat data as cyclical in SCC

-n <n> --lagn=<n> Lag gap in SCC. Default=1

-f --fold Fold uppercase letters to lower case

-t --terse Terse output

-h or -u --help Print this text

It handles two formats - Hex (the default) and Binary. In hex mode, the hex is converted to binary on the fly before being tested. It strips non hex characters and 0x prefixes. The symbol size is independent of the representation. It defaults to 8 for hex mode and 1 for binary mode but can be set to be be from 1 to 32 bits. Larger symbol sizes are limited by the resulting 2^n occurrence count table.

E.G.

djent hexfile.hex

opening hexfile.hex as hex text

Symbol Size(bits) = 8

Shannon IID Entropy = 7.983000 bits per symbol

Optimal compression would compress by 0.212501 percent

Chi square: symbol count=10240, distribution=238.00, randomly exceeds 77.05 percent of the time

Mean = 127.863965

Monte Carlo value for Pi is 3.139508 (error 0.07 percent).

Serial Correlation = -0.014567

If you want to measure the properties of ascii data (not hex), then use binary mode with 8 bit symbols.

# djent -b -l 8 hexfile.hex

opening hexfile.hex as binary

Symbol Size(bits) = 8

Shannon IID Entropy = 4.052783 bits per symbol

Optimal compression would compress by 49.340206 percent

Chi square: symbol count=20800, distribution=303260.11, randomly exceeds 0.00 percent of the time

Mean = 57.392740

Monte Carlo value for Pi is 4.000000 (error 27.32 percent).

Serial Correlation = -0.008395

Multiple files can be fed in. Terse mode (-t) is useful to format the output from multiple files.

#djrandom -b -k 10 -m correlated --correlation=0.2 > corr0p2.bin

#djrandom -b -k 10 -m correlated --correlation=0.3 > corr0p3.bin

#djrandom -b -k 10 -m correlated --correlation=0.4 > corr0p4.bin

#djrandom -b -k 10 -m correlated --correlation=0.5 > corr0p5.bin

# djent -b -t corr0p*.bin

0, File-bytes, Entropy, Chi-square, Mean, Monte-Carlo-Pi, Serial-Correlation, Filename

1, 10240, 1.000000, 88.057350, 0.499738, 2.933177, 0.201680, corr0p2.bin

2, 10240, 0.999938, 0.808874, 0.504626, 2.769050, 0.300127, corr0p3.bin

3, 10240, 0.999975, 9.083176, 0.497046, 2.780774, 0.399825, corr0p4.bin

4, 10240, 0.999998, 66.992535, 0.499255, 2.698710, 0.500628, corr0p5.bin

3 bit symbols? Not a problem. Use the -l <n> directive.

# djent -b -l 3 binfile.bin

opening binfile.bin as binary

Symbol Size(bits) = 3

Shannon IID Entropy = 2.999628 bits per symbol

Optimal compression would compress by 0.012417 percent

Chi square: symbol count=27304, distribution=14.18, randomly exceeds 4.81 percent of the time

Mean = 3.475974

Monte Carlo value for Pi is 3.139508 (error 0.07 percent).

Serial Correlation = 0.001389

Post has attachment

Post has shared content

Public

I suspect I'm not the first person to think a cloud connected keyboard has to be a really, really bad idea. It's hard enough keeping your keystrokes private on a USB connected keyboard.

A key board with Alert Keys #sysaadmin #technology

Post has attachment

I've completed and tested the last (I.E. the 16 Ohm version) of the -20dB power attenuator designs. You can get it here: http://www.blacksheepelectronics.com/store/p5/16_Ohm%2C_25W%2C_-20dB_Guitar_Speaker_Attenuator.html. So 4, 8 and 16 Ohm versions are available. Let me know if I've missed some obscure brand that uses another impedance, but after trawling the specs of many web sites, I think that covers all of them.

Post has attachment

Another little article on power attenuators, with math, because we all love math. http://www.blacksheepelectronics.com/how-does-an-attenuator-work.html

Post has attachment

So I made a power attenuator for my guitar. I found my Mesa Mark V 25 guitar amp way too loud to play at home. The gain and master knobs had to be down at 0.1, rather than 11 for it to be quiet enough to use at home. I got an attenuator and it was awful, messing up the tone and with a linear instead of logarithmic knob so its only useful knob position was in the first 1% of the knob's motion. So I build a better, flat frequency response, fixed power attenuator to sit between the amp head and the speaker cabinet. It worked so well I thought I might sell a few to other guitarists who have the same problem with their home amp. I've made them available at:

http://www.blacksheepelectronics.com.

They're all capable of handling up to 25W. I'm working on the design for a higher wattage one, but the thermal design is more challenging. They're hand built by me. So I can't make a huge number immediately. I've made 4 and 8 Ohm versions. I've got parts on order for a 16 Ohm version that'll be available within a couple of weeks for people with that little Orange head with the 16 Ohm output.

http://www.blacksheepelectronics.com.

They're all capable of handling up to 25W. I'm working on the design for a higher wattage one, but the thermal design is more challenging. They're hand built by me. So I can't make a huge number immediately. I've made 4 and 8 Ohm versions. I've got parts on order for a 16 Ohm version that'll be available within a couple of weeks for people with that little Orange head with the 16 Ohm output.

‹

›

12/28/15

7 Photos - View album

Post has attachment

My thoughts after inspecting a budget guitar recently.

Post has attachment

I noticed this doorway while at the ICMC 2015 conference.

Post has attachment

Some disks found in the back of the garage.

Wait while more posts are being loaded