Profile

Cover photo
Michael Murphy
Worked at Howard Memorial Hospital
Lives in Ashburn, Virginia
475 followers|1,032,690 views
AboutPostsCollections+1'sReviews

Stream

Michael Murphy

Shared publicly  - 
 
The performance issue with AMD processors has been resolved. It would seem that Linux distributions have transparent_hugepages set to always, which causes a 1000% overhead increase. You can set this to a better value, madvise, by running the following command:

sudo sh -c "echo madvise > /sys/kernel/mm/transparent_hugepage/enabled"

https://github.com/rust-lang/rust/issues/36705#issuecomment-249713259
As I've developed this Parallel application, I've noticed that everyone who uses the application with an AMD processor always reports significantly slower runtime performance than those using Intel...
4
1
Michael Murphy's profile photoIkey Doherty's profile photoLinux Lady05 Winter's profile photo
5 comments
 
+Ikey Doherty it's a real pain. I got it running but still tinkering. 
Add a comment...

Michael Murphy

Shared publicly  - 
 
There's apparently an issue with the Rust compiler producing multi-threaded code that runs insanely slow on AMD hardware (slower than interpreted languages), and yet blazing fast on Intel hardware. I'm not exactly sure what's causing it, but you can see from these two sysprof reports that there's something going seriously wrong with the AMD system on the right (AMD FX-4300) compared to the mobile i5 2410 @ 2.3 GHz on the left.
1
Add a comment...

Michael Murphy

General Discussion  - 
 
 
This is some seriously crazy hardcore optimization work in Rust, for something so trivial. Source code is pretty insane to attempt to decipher what's going on.

Source: https://github.com/llogiq/newlinebench/blob/master/src/lib.rs#L169:L234

Blog: https://llogiq.github.io/2016/09/24/newline.html
Llogiq on stuff. Counting Newlines Really Fast. 24 September 2016. Looking into xi editor's rope crate, I found a method with a nice nerd-snipey (obligatory XKCD) comment, which I will reproduce here in full so you don't have to chase the link and find back: // TODO: explore ways to make this ...
View original post
2
Add a comment...

Michael Murphy

Shared publicly  - 
 
I will have a new release of Parallel released soon, after I am finished refactoring and writing unit tests. It may now process an infinite number of input arguments as the bug with child process handles sticking around has been resolved, memory consumption has been reduced to 1/5th that of GNU Parallel thanks to a disk buffering technique. According to benchmarks with detailed statistics provided by time, my implementation is currently 40x faster than GNU Parallel (11x when comparing wall time), but there is room for improvement to make this even faster:

Rust Parallel

~/D/parallel (master) $ seq 1 10000 | time -v target/release/parallel echo > /dev/null
Command being timed: "target/release/parallel echo"
User time (seconds): 0.48
System time (seconds): 2.48
Percent of CPU this job got: 59%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:04.93
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 12928
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 2198164
Voluntary context switches: 73174
Involuntary context switches: 36678
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0

GNU Parallel

~/D/parallel (master) $ seq 1 10000 | time -v parallel echo > /dev/null
Command being timed: "parallel echo"
User time (seconds): 97.04
System time (seconds): 29.17
Percent of CPU this job got: 232%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:54.17
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 66848
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 15070207
Voluntary context switches: 250452
Involuntary context switches: 113320
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
1
Add a comment...

Michael Murphy

Shared publicly  - 
 
Automated command-line argument parsing in Rust is getting rather advanced. It has quite the feature set and customizability.
Standard Argument Types. clap supports all standard argument types, flags/switches, options, free/positional arguments, and even subcommands! Here's some extra information about special features clap supports. Both short and long versions supported ( -f and --flag respectively); Combining short ...
1
1
Add a comment...

Michael Murphy

Shared publicly  - 
 
Not that I would advise it, but it would appear that it is now possible to use inline C++ in Rust due to the work by yet another Google employee using Rust during their 20% time. This is made possible by compiler plugins and Rust's macro system. The inline_cxx! and cxx! macros make it possible to write C++ natively in Rust, but the rust! macro additionally allows you to inline Rust into the inlined C++. Thus, Rust now has the capability of allowing you to write Assembly, C++ and Rust at the same time.
rustcxx: Using C++ from Rust made easy
3
1
Add a comment...

Communities

14 communities

Michael Murphy

Shared publicly  - 
 
Very useful information. Demonstrates how to create and import dynamic Rust libraries in Rust projects, which allows support for dynamically-loading plugins in your Rust program, or allowing for Rust software to be written to share dynamic libraries with other Rust software.

https://damienradtke.com/post/rusty-dynamic-loading/
1
Add a comment...

Michael Murphy

Shared publicly  - 
 
After two weeks of work, I've finished piecing together my next release of Parallel. Huge changes here, so the git commit is rather large: https://github.com/mmstick/parallel/

Here's the current status of the codebase, as told from the point of view of the Tokei line counter.

Before

-------------------------------------------------------------------------------
Language Files Lines Code Comments Blanks
-------------------------------------------------------------------------------
Rust 8 1408 1112 149 147
-------------------------------------------------------------------------------
Total 8 1408 1112 149 147
-------------------------------------------------------------------------------

After

-------------------------------------------------------------------------------
Language Files Lines Code Comments Blanks
-------------------------------------------------------------------------------
Rust 16 2049 1645 180 224
-------------------------------------------------------------------------------
Total 16 2049 1645 180 224
-------------------------------------------------------------------------------

So what has changed? I have created a disk buffering mechanism that will buffer input arguments to and from the disk as efficiently as possibly using a fixed-sized 8K byte array. This was then used as a building block to create an input argument Iterator that is placed inside of an Arc<Mutex> and shared among all the threads, which then grabs inputs from the iterator until no more arguments exist.

Additionally, errors are now being properly logged to an error log, whereby the parallel command will now re-print the errors after execution is finished to remind the user that some errors occurred during processing. Parallel will no longer stop when an error occurs as well.

So there's a number of features that I will be working for next, and I have a to-do list that highlights the important ones: https://github.com/mmstick/parallel/blob/master/TODO.md
2
Add a comment...

Michael Murphy

Shared publicly  - 
 
This is some seriously crazy hardcore optimization work in Rust, for something so trivial. Source code is pretty insane to attempt to decipher what's going on.

Source: https://github.com/llogiq/newlinebench/blob/master/src/lib.rs#L169:L234

Blog: https://llogiq.github.io/2016/09/24/newline.html
Llogiq on stuff. Counting Newlines Really Fast. 24 September 2016. Looking into xi editor's rope crate, I found a method with a nice nerd-snipey (obligatory XKCD) comment, which I will reproduce here in full so you don't have to chase the link and find back: // TODO: explore ways to make this ...
2
1
Add a comment...

Michael Murphy

General Discussion  - 
 
 
Automated command-line argument parsing in Rust is getting rather advanced. It has quite the feature set and customizability.
Standard Argument Types. clap supports all standard argument types, flags/switches, options, free/positional arguments, and even subcommands! Here's some extra information about special features clap supports. Both short and long versions supported ( -f and --flag respectively); Combining short ...
View original post
4
Add a comment...

Michael Murphy

General Discussion  - 
 
 
Not that I would advise it, but it would appear that it is now possible to use inline C++ in Rust due to the work by yet another Google employee using Rust during their 20% time. This is made possible by compiler plugins and Rust's macro system. The inline_cxx! and cxx! macros make it possible to write C++ natively in Rust, but the rust! macro additionally allows you to inline Rust into the inlined C++. Thus, Rust now has the capability of allowing you to write Assembly, C++ and Rust at the same time.
rustcxx: Using C++ from Rust made easy
View original post
7
Add a comment...

Michael Murphy

Shared publicly  - 
 
I've spent the past five days or so working on a rather elaborate method of accepting an infinite number of input arguments to my Parallel program. I'm estimating that it could take me up to another week or so to have everything perfected. The current implementation will quickly run out of memory if there's a large number of arguments that exceeds the amount of memory that you have, but I want to retain the ability to know how many total input arguments that will be processed, which requires first reading all of the input arguments in the beginning of the program.

As I want my solution to be as fast and as memory-efficient as possible, I've opted out from Rust's higher level features, such as Vectors, and instead have been creating my own 'DiskBuffer' data structures which operates entirely with fixed-size arrays and gradually building up higher level features from there. The design is still a WIP though so I may make major changes to how it works as I gradually work towards completion.

The basic idea is to take input arguments and store them into a fixed-size byte array. The data structure responsible for keeping track of appending new bytes to the fixed-size array will automatically dump the contents of the buffer to an 'unprocessed' file on the disk when the buffer's limit has been reached, and then resume adding more input arguments into the buffer until all input arguments have been processed. Then, a higher level data structure will be created, which implement's Rust's Iterator trait, and will buffer N amount of bytes at a time from the disk and keep track of the current state of the iterator and the segment of inputs that are currently stored in the internal buffer.

Once everything's working perfectly, I'll fiddle with the size of the fixed size arrays to find an optimal buffer size, possibly implement compression using the fastest compression/decompression method available (either zstd which will incur a non-Rust dependncy, or the Rust implementation of Zopfli which will incur zero dependencies on the user), experiment with ways of making the threads/buffers faster, lots of refactoring and a ton of additional unit tests to ensure that Parallel is perfectly stable at any given point in time.
3
Add a comment...
Michael's Collections
Communities
14 communities
Story
Tagline
What do you see when you reverse the chessboard?
Introduction
I speak concisely, or not at all.
Places
Map of the places this user has livedMap of the places this user has livedMap of the places this user has lived
Currently
Ashburn, Virginia
Previously
Saratoga, Arkansas - Nashville, Arkansas - Hope, Arkansas
Contact Information
Home
Email
Collections Michael is following
Work
Occupation
Information Technology
Skills
Critical Thinking, Logical Problem Solving, Computer Programming: {C, C++, D, Go, Java, Julia, Python, Rust}, Shell Scripting, System Administration, Linux, Windows, Mac OS X, Computer Hardware and Software Repair, Computer Networking, Technical Writing, Scribus Desktop Publishing, GIMP Image Manipulation, Krita Image Manipulation
Employment
  • Howard Memorial Hospital
    IT Intern, 2010 - 2011
    Keeping records of all the available IP-connected devices, RDPing to Windows desktops and performing regular maintenance updates and upgrades, solving user errors, performing maintenance of printers, auditing computer and network equipment, installing new systems, computer repair, and managing networks.
Basic Information
Gender
Male
Relationship
In a relationship
Michael Murphy's +1's are the things they like, agree with, or want to recommend.
Glovebox - Side launcher – Android-Apps auf Google Play
market.android.com

Glovebox brings to android the best way of multi-tasking. Swipe without lifting the finger to select an application to open. You don't even

Text Mode
chrome.google.com

Browse the web without distractions via simple text based pages.

GIMP
plus.google.com

Free application for serious image manipulation

Scribus
plus.google.com

Free cross-platform desktop publishing (DTP) solution

Linux
plus.google.com

Linux - doing it right

Apple: They Didn't Build That - Ideas Apple Bought, Borrowed and Stole
www.bestmastersprograms.org

A look at the ideas Apple bought, borrowed and stole to create their most popular products.

I've been trying to find a proper Chinese restaurant ever since I moved from Southwest Arkansas and I have just been having a difficult time because all of the restaurants in the area are in poor quality across the board. You would think that the cities would be able to offer higher quality Chinese restaurants than small rural towns and villages, but that is not the case. How sad it is that when you purchase something as simple as sesame chicken, here and every other Chinese restaurant, that they deliver you a plate full of soggy mystery meat containing large amounts of unhealthy fat, and for obscene prices to boot. I'm used to going to a small, quiet restaurant in Nashville, Arkansas named the Chinese Buffet that would let you eat as much as you want for a low fee of, say $6 per person, that offers higher quality egg drop soup and chicken made with only the best parts of the chicken -- no fat, and no mystery meat, perfectly crispy all the way throughout and not soggy like Chau's Cafe. While it's common for Chinese restaurants throughout Southwest Arkansas and Texas to offer high quality food at low prices, apparently it's the exact opposite here.
• • •
Public - 5 months ago
reviewed 5 months ago
Coming from southwest Arkansas where Mexican restaurants are a dime a dozen, I'm going to have to place Uncle Julio's below the quality of the worst Mexican restaurant I have ever been to. Not only are the prices exhorbitant, such as charging $10 for a small bowl of queso whose actual market value is 50 cents, the menu is greatly lacking in quality Mexican food and the portions are miniature. I'm sorry, but you have nothing on small town restaurants like La Villa from Nashville, Arkansas. At least La Villa can offer a complete full course meal on a large plate for vegetarians at half the price of a bowl of a queso at this establishment, with fresher ingredients to boot.
• • •
Public - 5 months ago
reviewed 5 months ago
3 reviews
Map
Map
Map
This is one of the worst places you can possibly eat in Southwest Arkansas, especially if you are a vegetarian. Their menu is completely void of any vegetarian options besides a meager selection of vegetables offered as side dishes for exorbitant prices, some of which even contain meat. For $7, I ordered mashed potatoes, baked beans and french fries. What I received was nothing more than a handful of Walmart-quality french fries, a small portion of mashed potatoes and a tiny sauce dish of baked beans that literally contained half a cup of sugar mixed in with a few beans and BACON. Why on Earth a vegetable dish of baked beans contains bacon is beyond my comprehension. Even the sweet tea that I was served was nothing more than tea-flavoured syrup. Do people really stomach this amount sugar on a regular basis? As a result, I could not eat the baked beans, quickly finish the mashed potatoes in about four spoonfuls and ate the french fries within twenty seconds, despite waiting 20 minutes for such a small dish. My advice? Either cook a real meal at home for a seventh the cost or just heat up a bowl of noodles. You'll save money, it will taste better and you won't have to leave a restaurant still hungry. P.S. Their menu has bad typesetting which is really annoying to look at. You can tell that they used spaces rather than tabs so their menu items aren't aligned correctly.
• • •
Public - a year ago
reviewed a year ago