Shared publicly  - 
 
To clarify a few things regarding that "debug" mess:

I will change systemd to not log to kmsg if journald is around when "debug" is specified. We will continue to log to kmsg during the early bootphase however als long as journald is not running yet. At that point there is simply no other option for that, because persistent storage is not available yet at that point and we need som way for people to get their logs out of the system, and early kmsg is the only way really, since it is hooked up with netconsole, serial tty, crashkernels, ... and all that other stuff.

I also believe that it is the right thing for the kernel to ratelimit whatever userspace sends it. For the same reason as we have to ratelimit all log streams in journald too, regardless where they come from. If you get a stream of logs from a 'differently priviliged' component then you need to rate limit it, that should be common sense. Of course I wish the ratelimiting applied is always configurable for the user, so that he can control how much and how soon things get dropped.

For me it is out of question though that systemd and other core os components should continue to parse the 'debug' kernel cmdline option, and increase their debug levels then. Generic options like that are supposed to be useful for real people, and there's a long history of options like that which influence both kernel and userspace (quiet, root=, ...). We are putting together an OS here after all, not just a kernel, and a kernel is just one component of the OS among many, and ultimately an implementation detail. We are writing an OS here for the general purpose, not just a toy for a clique of kernel developers. Moreover, there are individual kernel cmdline options for both the kernel itself and systemd to control just their log levels, and nothing else. so if you want finegrained control, you already have it, 'debug' is just the simple option that groups them all under a single oneshot option. It's the option an admin can specify which tells him why the system doesnt boot, regardless whether the kernel or systemd is at fault or any other part of the core os involved in boot. Thats simply userfriendly.

I do not appreciate all that talk from certain parts of the community that the systemd folks need to 'get tought a lesson'. That turns this into some kind of power game, which I am totally not interested in. Comparing us to Putin, or trying to bind merging of completely unrelated code to us doing whatever some kernel devs want (which is blackmail at worst, and childish at best, and childish we can be on our own enough, thanks) will just make us ignore you, and that's it. I prefer to deal with technical questions, and intimidation is not a technical question.

what the various news outlets wrote about all this is quite ridiculous. It's sad how badly these sites report, quite a shame.

And that is all.
119
28
Cameron Wood's profile photoSteve Brown's profile photoThomas Eichberger's profile photoJames Nelson's profile photo
47 comments
 
The trouble is that Linux is not simply "an operating system". Operating systems may be built using the Linux kernel. What should not happen is to have any one component of a Linux-based operating system utterly dependent on any one module. The systemd and udev developers SEEM to want their projects to control and dictate the content of the whole. Systemd should be simply one of the options available and not the "only game in town" it's developers and fanbois seem to intend. 
 
The thing I don't understand is how 'debug' is in any way different from 'loglevel' - there's a 'loglevel' for the kernel, and a 'systemd.loglevel' for init 1 afaik. Diverting from this with 'debug' seems inconsistent?
 
I'm not really sure how your comment is relevent to the topic +Kirk Pearson. The fact that the +systemd project is implementing the core userspace building blocks needed for an operating system (which obviously runs on top of the Linux kernel) in no way precludes the Linux kernel being used in a different way by a different userspace. The fact that systemd developers focus on systemd-based operating systems (as much as android developers focus on android-based operating systems) shouldn't really be surprising...
 
+Colin Guthrie I refer to this part "I do not appreciate all that talk from certain parts of the community that the systemd folks need to 'get tought a lesson'. That turns this into some kind of power game, which I am totally not interested in. Comparing us to Putin, or trying to bind merging of completely unrelated code to us doing whatever some kernel devs want (which is blackmail at worst, and childish at best, and childish we can be on our own enough, thanks) will just make us ignore you, and that's it" which is rather the tail wagging the dog. K.S. even submits issues with his code in udev to the kernel devs for a patch? OMG the arrogance of the young lions. 
 
"For me it is out of question though that systemd and other core os components should continue to parse the 'debug' kernel cmdline option, and increase their debug levels then."

In the interests of clear communication, as a native English speaker, this strongly implies to me that your opinion is that systemd should not parse the "debug" option whereas the rest of your post appears to argue the opposite. The phrase "out of the question" is used to indicate a concept which is so entirely ludicrous it need not be discussed. It doesn't seem that this is the sense in which you used it.
 
"We will continue to log to kmsg during the early bootphase however als long as journald is not running yet. At that point there is simply no other option for that" - logsave(8), as noted by +Theodore Ts'o in the lkml thread and in his g+ thread. This way dmesg won't be spammed and everyone will be happy.
 
+Rich Wareham The German equivalent can be used in both ways.. quite weird. He probably meant "it's beyond debate..".
 
+Rich Wareham and +David Herrmann I think he meant 'beyond debate'.

It struck me as odd as well.
I'm Dutch and speak German as well so can appreciate where it came from.
 
It can be used in English both ways as well. Which is to say its still ambiguous.
 
In my dialect, "It is out of the question" always means "It's not going to happen."

Who knows, though. English varies region to region. I hardly know what I'm saying myself half the time.
 
It comes from the German "nicht im Frage", which - literally translated - is "not in the question", hence the 'mistake' of "out of the question".
 
Looks like we need ... wait for it ... Esperanto :-) 
 
Personally, I appreciate your work but feel like you're missing the point. Tactics only seemed to escalate when discussion failed. But that's just my one opinion.
 
+Tim JP It isn't uncalled for drama - these are important issues. If it requires drama for them to be addressed so be it. 
 
You are completely right on two points: 1) user space tools should parse command line options that are sensible to then. 2) kernel should rate limit user space process. 

That said. I see that things should not break. If systemd is breaking the system, that's a bug, therefore should be fixed. No questions asked. The subsequent refuses to acknowledge and fix bugs by systemd devs is the reason to everyone be pissed off.
 
+Cristian Rodriguez No worries there - many don't! The motivation behind my comments here and practically everywhere else is that systemd has been rammed down our throats for no good reason. The casual desktop user might say "no big deal" but some people are losing money and sleep over this in the data center. Our Free and Opensource communities are in a death spiral and don't even realize it.
 
+Kirk Pearson If you do not like systemd, why do you use it? Did Lennart and Kay put a gun to your head or something?
 
+Andreas Tunek I don't. My clients are not liking that they will be forced to. Slackware and Gentoo are the only two parent distros that have chosen not to use it. 
 
+Andreas Tunek To bring you up to date with the current state of affairs in enterprise Linux - There is Red Hat, Suse and Debian. All others are a trickle compared to a deluge. All of the above are going to systemd. That is the pressure. The solution is abandoning Linux for Solaris, BSDs or Windows. Losing customers means losing fees and donations. In the long run this hurts Linux and Free/Opensource.
 
+Kirk Pearson So, your clients had no problem with having sysvinit rammed down their throats? And have no problem with the linux kernel being rammed down their throats (or are you really one of the handfull of people using debian with a different kernel)?
 
+Kirk Pearson
So, in actuality, nobody is forcing anyone. If you want to use RHEL 7 you will have to use systemd (and rpm, glibc and loads of other stuff). Same for other distros. So you are basically worried that people are going migrate from linux/systemd based OS towards other systems. You might be correct about that, but I have not seen any data indicating that this is the case.

But maybe you could tell me? Why would anyone migrate from linux/systemd towards the other systems you mention? What kind of advantages to they have?
 
I can't really understand why so many people are complaining about systemd. IMHO it's a major improvement to the platform. These guys are innovating and improving things. Sometimes they're overbearing, they make mistakes, and they're not as considerate and some would like. Big deal. Stuff breaks sometimes when you innovate. You deal with it.

And that's maybe a thing they do need to learn a bit better; when you're doing these things, people tend to put everything you do under a microscope and whine loudly if things break. Best attitude is to quickly fix it in a way that is in line with your long term goals and satisfactory to the whiners.

So all in all I think they're doing a great job. systemd is very nice and improves sooo many thing a lot. It's a bit too far-reaching for my (current) taste but I'll adjust.
 
Just make sure that your solution is "compatible" with what Linus said. I think that you 2 guys are both right: systemd is too important for Linux to be thought oft just another user space component, so yes, i think you should be allowed to interpret the kernel's debug parameter, and yes, you should not be able to flood the kernel's ring buffer, but that's a thing the kernel should prevent by itself.
 
Doesn't Solaris have something similar to systemd?  In any case, I think your clients might just be used to using sysv init and do not want to learn new things.

If they want to move on to solaris or bsd that is their right, but for BSD there isn't any compelling reason to go there as it is relatively unchanging and doesn't have the kind of enterprise support that Linux has.
 
+Sriram Ramkrishna  in corporate world or BU does anybody learn anything new at all???? Plus who wants the pain to learn new thing and implemented in BU?? No manager can...no guys in suits can..they refrain from it..It's okay for us..because we live and drink with OSS and other stuff..BU not need it. My personal experience...correct me.
 
It depends of course.  I see both.  A lot of people in IT these days are 40+ because of the way teh market has been, it's mostly older people especially in teh senior positions.  So there is definitely a component of "I dont wnat to learn"

But hardware changes software and sometimes if we want to compete we need to upgrade and improve our eco-system.
 
It's about computers being used to do actual work. When I'm doing actual work, I don't like surprises. With systemd so close to the cutting edge, it's prone to surprises.

I think that systemd is an important innovation for GNU/Linux, but this drama illustrates how it's not all peace and cooperation like +Lennart Poettering keeps saying, but there's railroading like Ian Jackson says. It may be technically better, but it annoys people. That's why most efforts to improve things take forever.

And, yes, the press like to turn disagreements into major controversies. Intellectual laziness is good for business. They suck.
 
+Sriram Ramkrishna
 In switching from Redhat/Fedora to Funtoo in order to avoid systemd I'll be learning just as much or more than I would have by simply accepting systemd in my distro.   If you can't understand why some think that systemd's failure to observe "the UNIX way" makes it undesirable that's your failing.
 
+Lennart Poettering but I should rather say +Kay Sievers, why not just be kind to other developers and acknowledge your limits? Nobody is perfect, we are okay with that. Everybody makes mistakes, and that is just one. It happens, just move on and learn to say sorry when needed. 
 
+David Herrmann No, it cannot be used in both ways. The mistake here was to translate "außer" with "out of" while he should have used "beyond". This little mistake translates back to "indiskutabel" instead of "außer Frage stehen". You simply have to translate phrases as phrases not word by word... ;-)

Also in German, the phrase "außer Frage stehen" means "it is obvious", "es ist gewiss / klar / steht fest", and not the opposite. It is "out of" instead of "beyond" which reverses the meaning. I'm not a native speaker but this is how I understand both phrases.
Translate
 
Systemd isnt core component. System can boot and work fine without it. And it should be simple and easy. However in current state systemd looks like a bycycle on scrappers. 
 
I don't judge the technical qualities of one init implementation versus the other, but I evaluate the project on the basis how some developers, especially +Lennart Poettering and Kay Sievers (but I know exceptions), are handling bugs and feedback. They prefer to ignore or ridicule proper bug reports (see the initial bug report of the debug case), without answering properly. Same happened with the CGROUP=n gives you a crashing PID1.

We are talking about a project that is high up in the hierarchy, not some leaf program like an editor. Developers of those upper-level (in terms of close to the inner workings of the system) have to take up a certain level of responsibility, higher than the one of leaf-devs.

Unfortunately it seems that the two main devs mentioned here are not up to this responsibility, and prefer to bitch around and make fun of bug reporters.

I can only hope that with the proliferation of systemd in the big distros, feeling of responsibility and response patterns improve, instead of deteriorating even further.
 
+Norbert Preining If they are not "up to this responsibility" there is a simple solution for that : Write something better, convince the complete ecosystem to adopt your tools, find people that you think are suitable for your effort.. or deal with it like everybody else.
 
+Cristian Rodriguez point taken in the same way as I expressed my opinion above. This is exactly the attitude I am criticizing. I do other development work, and my competences are in a different area. That does not exempt me from the right to utter well-founded criticism. That relates very nicely to what +Theodore Ts'o said about the attitude of systemd and for that matter, GNOME, developers in https://plus.google.com/117091380454742934025/posts/4W6rrMMvhWU "go away, you're clueless, we know better than you, and besides, we have commit privs and you don't, so go away"

Yes, this is what I call "not taking up responsability", and you just did nothing else but support this opinion. Thanks for that, very much appreciated ;-)
 
+Tim JP Really. I look at https://www.archlinux.org/ and the third-newest news item is about how systemd now ignores a long-standing traditional configuration file. That can count as a surprise. I run Arch on my Pogoplug, so I can't just put a Fedora DVD on that thing. I'm still not used to the systemd way of doing things.

I see other articles like http://lwn.net/Articles/555920/ and http://lwn.net/Articles/574317/ and I wonder exactly how stable systemd can be.

At this stage, from the outside, I'm not terribly worried yet. Sure, Lennart and Kay, and Ulrich Drepper before them, can be difficult to work with. Important work is still being done.
 
+Tim JP
makes them want to use Windows?  ::shudder::  ::wince::  Never.  Not ever.
 
systemd is terrible plague .. the most ridiculous thing of systemd is that it killed great ideas and must-have features by encrusting them to semi-mandatory crap around .. systemd developers are idiots (sorry) who just overrate theirs own skills and work (aka software) .. I completely agree with this blog post: https://lkml.org/lkml/2014/8/12/459 and I hope, that someone sane forks soon systemd to systemd-tiny (or something similar) project dropping all that shit around and laughing loud to the lennart and kay's faces
 
+Lennart Poettering “Hmm. I am not sure where you are taking the "doesn't play well with others" from. Our community these days is huge and friendly and we work together very well within the community and with the various projects outside of the community.”
http://lwn.net/Articles/621015/

Do you have any idea how tone-deaf that statement is? Of course you’re working well within your community. Your interactions outside your community are more of a problem.

For example, you’re on record as being dismissive of POSIX compatibility. Everybody who’s currently in the systemd community had to accept that. But Ian Jackson is very invested emotionally in having Debian on other kernels for some reason. This is an irreconcilable difference.
https://archive.fosdem.org/2011/interview/lennart-poettering

And I think a bunch of people have gotten used to their old way of lazily interacting with the separate groups for udev, dbus, logging, and whatever, and don’t feel good when keeping up with the latest updates for any of those means switching to a single git repository, all or nothing. I know you keep saying that people have a choice, they don’t need to install every program in the repository, but I think it’s a false choice when someone just wants the latest udev, and finds that that means they need to download systemd and also install the latest intltool just to get the darn thing to compile, and then udev has changed something so a configuration that used to work doesn’t work anymore… And you’ve been in systemd since the beginning, so you know how it works, but a (potentially graybeard) beginner will see the now 21-part systemd for administrators series, and that doesn’t feel good at all.

I think you have the best of intentions and a lot of skill; I’m also rather skeptical of POSIX as the end-all be-all of API design, and Linux needs to move forward. But I think you have a public relations problem, and I think it’s ironic that you blame Linus for the attitude problem when people feel the same way about you. And especially about Kay Sievers.
Add a comment...