Profile

Cover photo
darken
13,438 followers|1,392,057 views
AboutPostsCollectionsPhotosYouTube

Stream

darken

Shared publicly  - 
 
v4.3.5 is now on the way to become the next production version.
(Currently in staged rollout)

Update range v4.3.3 to v4.3.5 brings a new storage detection and binary (busybox/toybox) setup system. The goal for these changes is to improve SD Maids compatibility with with uncommon devices. It also makes SD Maid more robust in dealing with the increasingly diverse market of SuperUser applications.

The Explorer does now support archive extraction on external sdcards and AppControl allows selection of multiple filter tags (e.g. frozen + system).

http://sdmaid.darken.eu/download
http://sdmaid.darken.eu/changelog
http://sdmaid.darken.eu/issues

Beta discussions:
* v4.3.3 https://plus.google.com/u/0/b/116634499773478773276/116634499773478773276/posts/3CoX2VD4pxE
* v4.3.4 https://plus.google.com/u/0/b/116634499773478773276/116634499773478773276/posts/4sQLqnC5SiB
* v4.3.5 https://plus.google.com/u/0/b/116634499773478773276/116634499773478773276/posts/4vH56P5yeph
19
Jost Schwider's profile photodexter lords's profile photoCraig D's profile photoBrien J's profile photo
4 comments
Brien J
 
,h 
Add a comment...

darken
owner

General Discussion  - 
 
SD Maid v4.3.4 took a lot of work for relatively few changes.

A few crash fixes but mostly a new way of setting up busybox/toybox binaries.

http://sdmaid.darken.eu/download
http://sdmaid.darken.eu/changelog
http://sdmaid.darken.eu/issues

Let me tell you a story about developing a root app:
Since some 5.0+ Android version SELinux forbids apps to run binaries from their own folder in /data/data with root permission. This sucks because some apps need to run it's binary from /data/data with root (including SD Maid).
What now? A bit of research showed that this restriction only exist for certain places, copying SD Maids binary with root into /system would work, but copying binaries into /system is uncool though modifying /system screws with OTA updates and other things.
Although a rooted device likely modified /system already it would be way cleaner if SD Maid just wasn't part of that equation. A bit more research showed that there is also no restriction for binaries placed in the RootFS (e.g. /strawberry), as the RootFS sits in RAM and gets wiped on reboot, this would also mean any changes SD Maid makes there are gone after a reboot, yay that's clean! So i wrote a system that tries different approaches to get this to work:

* Try SD Maids internal path (the default app files)
* If that fails we check if we can just use something pre existing on the device
* Can't use default and nothing existing allows us to work so we try RootFS, by first checking whether a systemless su install already created a location in RootFS, otherwise we make our own
* If that fails too we can only resort to placing our binary into /system which is the last resort effort before giving up and relinquishing root access

This all worked nice and well for a long time, until enough bug reports collected in the crash tracker to become noticeable. The reports said SD Maid couldn't detect any storages at all. People to so much weird stuff with rooted devices and I didn't think this was a bug for a while. Initially I was pretty stumped on the causes, but through a few users that made the effort to mail me about this I figured it out. Not all devices/rom combinations allow executing a binary from RootFS without root! Oh right, makes sense. So I just made SD Maid use root for this and published v4.3.1.

All was well in SD Maid world until I started to get mails about SD Maid showing no storage information for the sdcard or not detecting it all. sigh But thanks to the effort of a few users, supplying debug logs, this was quickly figured out. Depending on the superuser application (and superuser application settings, e.g. SuperSUs "Mount namespace separation") and your ROM, looking up mounts as root returns different results than looking them up without root. To be honest here, i knew about this behavior, but forgot about it... (and didn't notice it because I used SuperSU with "Namespace separation" turned on for testing). So the problem is easy enough to fix right? Call it once with root and once without, then merge the results.... Well, remember the above? There are devices where binaries can't be run from our apps folder with root and can't be run without root from RootFS. Sure we could jam the binary into /system which allows both user and root to access it but as mentioned, this is pretty dirty and SD Maid is all about neat & tidy, right?
So I thought this can't be too common, let's just add a fallback method that tries to use the default mount applet from /system, while they usually don't work very well to remount stuff, just listing the mounts should reliably work... and v4.3.3 was published.

But.... nope. Trying the system's native mount applet wasn't reliable at all, not even just for listing mounts. This was aggravated by SD Maids not calling applets directly, but by using a list of applets that were checked for compatibility during setup and this checking was quite harsh (as it should be though) and also ruled out most native mount applets.
v4.3.4 enters the stage. I bit the bullet and rewrote every line of code that uses a binary, to ask for a specific binary (user/root) and rewrote SD Maids setup routine to allow setting up two different binaries in two different locations if that means avoiding modifying `/system`. The new routines setup binaries preferring a single binary over multiple while opting for the least invasive way that allows SD Maid to work with root on the device (own-locations > pre-existing > rootfs_systemless > rootfs_custom > system_inject).
Oh my, what don't I do to keep your devices neat & tidy :).

If this finally fixes the storage detection / mount detection issues, it can go into production.
34
Ivan Krušlin's profile photoNasir Ahmed's profile photo
2 comments
 
Well done. 
Add a comment...

darken

Shared publicly  - 
 
SD Maid v4.3.2 is a major update and now the latest production version.
It's currently in staged rollout.

If you can't wait: http://sdmaid.darken.eu/download
Want details? Look here: http://sdmaid.darken.eu/changelog
For bugs or ideas, check: http://sdmaid.darken.eu/issues

Here are some notes:

* Tools will now display 'allocated size' instead of 'apparent size'. Allocated size is the size an item actually takes up on your storage (i.e. a 1 byte file actually uses 4096 byte on storage). Apparent size is a files length, how large the file itself tells us it is (i.e. a file may say it is 8096 byte long, but actually only takes up 4096 byte). A big source for discrepancy between sizes are caused by "sparse-files", these are files that contains a lot of 0000000. If the file system supports it a sparse file of length 1GB could be saved on a 1MB storage. I think we are mainly interested in size on our storage (allocated size), not how much data something uses when sending it over network (apparent size). This will be most noticeable with the AppCleaner (which will show larger amounts because of many small files) and the SystemCleaner (which will show smaller ammounts, due to thumbnails often being sparse files). The Explorer will show both sizes if they differ, format 'allocated size (apparent size)'.

* SD Maid now ships with toybox instead of busybox. Toybox is smaller and it fixes an issue I had with busybox. It's debatable whether toybox is faster, but looking at it's code it is at least a lot leaner. Toybox is also something that all future Android versions will ship which gives us a lot of room for good compatibility on future Android ROMs (who knows what Android 8.0 will throw our way). But why not both? SD Maid is now compatible with both toybox and busybox, and will fallback to a busybox on your system if that is what it takes to run without issue on your device. Maxium compatibility :)!

* If you have root SD Maid will no longer ask for secondary storage permission. There were too many problems with weird (rooted) ROMs that did not manage to grant the permission. You can still grant it manually via advanced settings. In theory, i thought performance should be better with secondary storage permissions through the storage access framework, but I couldn't actually measure it. ¯\(ツ)/¯

* You will probably never see the 'BUSYBOX error' page again because this only happens with root and SD Maid will now give up root if it would otherwise mean showing the error and giving up.

* AppControl doesn't load app sizes by default anymore and is thus much faster (see settings). Also has improved filtering options (right side drawer).

* The tool "Biggest" is now called "Storage analyzer" and has been completely rewritten.

* Explorer now supports unpacking any type of 'zip' files (.zip, .apk etc) (currently only from primary public storage).

* The LastModified tool is gone (bye!), and the Searcher now has options for minimum and maximum age (hi!).

* There is a new (better?) clutter reporter tool which uses GitHub. Let me know how it works and what I can improve, clutter reports is what gets us closer and closer to a 100% detection rate. To be able to click the questions marks in the Explorer, enable experimental settings.

* Besides dozen of bugfixes, there have also been multiple core changes that attempt to improve performance of SD Maid by reusing objects and making data lookups faster.

35
1
Debopriyo Basu's profile photoHshdjdjdhd's profile photoCraig D's profile photoJason Strachan's profile photo
6 comments
 
Thanks Darken!! I love your apps.
Add a comment...

darken
owner

General Discussion  - 
 
Weekend beta v4.3.0 brings a truckload of core improvements.

Here are the links if you just want to dive in:

http://sdmaid.darken.eu/download
http://sdmaid.darken.eu/changelog
http://sdmaid.darken.eu/issues

Reported issues:
https://github.com/d4rken/sdmaid-public/issues?utf8=%E2%9C%93&q=is%3Aissue%20milestone%3Av4.3.1%20
If your bug is not listed there, then I probably don't know about it ¯\(ツ)/¯.

Some notes:
* Tools will now display 'allocated size' instead of 'apparent size'. Allocated size is the size an item actually takes up on your storage (i.e. a 1 byte file actually uses 4096 byte on storage). Apparent size is a files length, how large the file itself tells us it is (i.e. a file may say it is 8096 byte long, but actually only takes up 4096 byte). A big source for discrepancy between sizes are caused by "sparse-files", these are files that contains a lot of 0000000. If the file system supports it a sparse file of length 1GB could be saved on a 1MB storage. I think we are mainly interested in size on our storage (allocated size), not how much data something uses when sending it over network (apparent size). This will be most noticeable with the AppCleaner (which will show larger amounts because of many small files) and the SystemCleaner (which will show smaller ammounts, due to thumbnails often being sparse files). The Explorer will show both sizes if they differ, format 'allocated size (apparent size)'.

* SD Maid now ships with toybox instead of busybox. Toybox is smaller and it fixes an issue I had with busybox. It's debatable whether toybox is faster, but looking it's code it is at least a lot leaner. Toybox is also something that all future Android versions will ship which gives us a lot of room for good compatibility on future Android ROMs (e.g. if we have to use the system toybox binary instead of SD Maids own). Busybox was not installed in most ROMs. Why not both? SD Maid is now compatible with both toybox and busybox, and will fallback to a busybox on your system if that is what it takes to run without issue on your device. Maxium compatibility :)!

* If you have root SD Maid will no longer ask for secondary storage permission. There were too many problems with people on chinese (rooted) ROMs that did not manage to grant the permission. You can still grant it manually via advanced settings. In theory, i thought performance should be better with secondary storage permissions through the storage access framework, but I couldn't actually measure it. ¯\(ツ)

* You will probably never see the 'BUSYBOX error' page again because this only happens with root and SD Maid will now give up root if it would otherwise mean showing the error and giving up.

* AppControl doesn't load app sizes by default anymore and is thus much faster. Also has improved filtering options (right side drawer).

* The tool "Biggest" is now called "Storage analyzer" and has been completely rewritten.

* Explorer now supports unpacking any type of 'zip' files (.zip, .apk etc) (without root for now).

* The LastModified tool is gone (bye!), and the Searcher now has options for minimum and maximum age (hi!).

* There is a new (better?) clutter reporter tool which uses GitHub. Let me know how it works and what I can improve, clutter reports is what gets us closer and closer to a 100% detection rate.

* Besides dozen of bugfixes, there have also been multiple core changes that attempt to improve performance of SD Maid by reusing objects and making data lookups faster.

All unit tests, I currently have, passed, but due to the amount of changes and changes to the core structure, I expect a few bugs, so this is a beta only release. Catching them and adding more tests is what a beta is for though, right :)? (give the issue tracker some love!)
40
1
Jay M's profile photoMaximiliano Oropeza's profile photodarken's profile photo
19 comments
darken
 
+Maximiliano Oropeza I think this is fixed, https://github.com/d4rken/sdmaid-public/issues/475 wait for the next update later today.
Add a comment...

darken
owner

Bugs || Ideas  - 
 
Got some reports that SD Maid changes the ringtone to default on the new Galaxy Note7. Does anyone have a Note7 and can tell me which tool/file causes this?
5
Add a comment...

darken
owner

General Discussion  - 
 
I want to rename the "Biggest" tool.
Other translations already use different synonyms so this mainly affects the english translation.

Thoughts?
195 votes  -  votes visible to Public
Storage usage
18%
Storage analyzer
61%
Storage use
7%
Storage utilization
9%
Storage capacity
5%
12
Stefan Blumoser (carabinieri5675)'s profile photoAdam C (adrok76)'s profile photoDamiano Orengo's profile photoStefan Wimmer's profile photo
18 comments
 
"Storage usage" looks like the best translation but sounds a bit bumpy ;-)
What about "Occupied (storage|space)"?
Add a comment...

darken
owner

General Discussion  - 
 
I'll merge the LastModified tool into the Searcher tool.

https://github.com/d4rken/sdmaid-public/issues/466

Thoughts on that?
14
1
Jay M's profile photoHerschel Brown's profile photoJari Beguš's profile photo
3 comments
 
I don't think I've used the Searcher or LastModified once lol
Add a comment...

Communities

darken
owner

General Discussion  - 
 
v4.3.5, just fixes ¯\(ツ)/¯.

This mainly fixes this UI related bug:
https://github.com/d4rken/sdmaid-public/issues/525
I thought it was a rare issue, but it's not, so the user impact was to huge to leave it for the next update.

http://sdmaid.darken.eu/download
http://sdmaid.darken.eu/changelog
http://sdmaid.darken.eu/issues

The mount/storage detection issue I talked about v4.3.4 seems to be fixed. Yay :).
23
Add a comment...

darken
owner

General Discussion  - 
 
SD Maid v4.3.3 is currently seeing the light of day.

Mostly bugfixes, but still a few new goodies.

Notes:
* The 0/0 size detection for some locations should be fixed now. It was related to mountspace seperation (root sees different mountpoints than the app user).
* Thanks to a users help "Prestigio" devices should have much better compatibility now, allowing for correct storage detection and granting of permissions.
* ZIP extraction should now support external storages on 5.0+ devices (and you can now actually cancel extraction).
* AppControls right-side filter drawer now supports selecting multiple tags (e.g. running+system), got ideas for filter tags? Make a ticket :p.

http://sdmaid.darken.eu/download
http://sdmaid.darken.eu/changelog
http://sdmaid.darken.eu/issues

If nothing comes up this can go into production.

On a side note, use of the issue tracker is increasing and I'm very happy about that.
29
Ray Hollingsworth's profile photoMauro Vieira's profile photo
2 comments
 
Obrigado 
 ·  Translate
Add a comment...

darken
owner

General Discussion  - 
 
Release canidate v4.3.2 fixes a few issues that came up in v4.3.1.

http://sdmaid.darken.eu/download
http://sdmaid.darken.eu/changelog
http://sdmaid.darken.eu/issues
29
2
darken's profile photo
darken
+
1
2
1
 
v4.3.2 is now considered stable and going into rollout.
Add a comment...

darken
owner

General Discussion  - 
 
Beta v4.3.1 gets us ready for a release.

This update fixes a crash when trying to run any tool on some devices (crash during initialization) and a few UI related crashes in the StorageAnalyzer tool.
Also see: https://github.com/d4rken/sdmaid-public/milestone/4?closed=1

v4.3.0 was already pretty stable and v4.3.1 is now a release canidate.

http://sdmaid.darken.eu/download
http://sdmaid.darken.eu/changelog
http://sdmaid.darken.eu/issues

24
Add a comment...

darken
owner

General Discussion  - 
 
I've turned of API of the old clutter reporting tool.
I will still process any remaining reports though (167 left).

SD Maid v4.3.0 will have a new (better) reporting tool that is more comfortable to use for both you and me.
Starting a report from inside the app will open SD Maids public GitHub repository and prefil a new issue ticket with all necessary information. You just have to hit submit.

While this will require you to have a GitHub account (they are free) this will also mean you can have recognition for your contributions.

My self written "report system" just wasn't up to the task and the good reports were drowned out by people using the comment field for crazy and insulting stuff. Writing somekind of anti spam system was out of the scope of that. Reports via GitHub will reduce this due to requiring an account, while giving me tools to ban such spammers.

The screenshot shows the reporting window and this is how such a generated report looks like:
https://github.com/d4rken/sdmaid-public/issues/468
(Content doesn't make sense, it's just a test)

45
2
Chuck Barnett's profile photoPablo Jimenez's profile photoMichael H.'s profile photoDamiano Orengo's profile photo
4 comments
 
You're great!!!
Add a comment...
darken's Collections
Communities
Contact Information
Contact info
Email
Story
Tagline
I enjoy Android development!
Introduction
Creator of SD Maid and other apps.