45

I have two CentOS 5 servers with nearly identical specs. When I login and doulimit -u, on one machine I getunlimited, and on the other I get77824.

When I run a cron like:

* * * * * ulimit -u > ulimit.txt

I get the same results (unlimited,77824).

I am trying to determine where these are set so that I can alter them. They are not set in any of my profiles (.bashrc,/etc/profile, etc.). These wouldn't affect cron anyway) nor in/etc/security/limits.conf (which is empty).

I have scoured google and even gone so far as to dogrep -Ir 77824 /, but nothing has turned up so far. I don't understand how these machines could have come preset with different limits.

I am actually wondering not for these machines, but for a different (CentOS 6) machine which has a limit of1024, which is far too small. I need to run cron jobs with a higher limit and the only way I know how to set that is in the cron job itself. That's ok, but I'd rather set it system wide so it's not as hacky.

Thanks for any help. This seems like it should be easy (NOT).


EDIT -- SOLVED

Ok, I figured this out. It seems to be an issue either with CentOS 6 or perhaps my machine configuration. On the CentOS 5 configuration, I can set in/etc/security/limits.conf:

* - nproc unlimited

and that would effectively update the accounts and cron limits. However, this does not work in my CentOS 6 box. Instead, I must do:

myname1 - nproc unlimitedmyname2 - nproc unlimited...

And things work as expected. Maybe the UID specification works to, but the wildcard (*) definitely DOES NOT here. Oddly, wildcards DO work for thenofile limit.

I still would love to know where the default values are actually coming from, because by default, this file is empty and I couldn't see why I had different defaults for the two CentOS boxes, which had identical hardware and were from the same provider.

Totor's user avatar
Totor
3,0683 gold badges26 silver badges32 bronze badges
askedFeb 5, 2012 at 4:08
nomercysir's user avatar
6
  • 3
    Do you have anything in/etc/security/limits.d/ ?CommentedFeb 5, 2012 at 4:40
  • No, that dir is emptyCommentedFeb 5, 2012 at 5:02
  • 1
    You can post the answer as an actual answer after a certain waiting period.CommentedFeb 5, 2012 at 12:49
  • 2
    I once looked this up somewhere. The defaults are set by the kernel. Partly hard-coded, partly dependent on the available ram. I think I found that on Oracle Metalink in the context of setting up SLES10 for Oracle-DB 11.2CommentedFeb 9, 2012 at 21:55
  • 1
    Could this question be marked as solved?CommentedAug 16, 2012 at 15:14

9 Answers9

61

These "default" limits are applied by:

  • theLinux kernel atboot time (to theinit orsystemd process),
  • inheritance, from the parent process' limits (atfork(2) time),
  • PAMwhen the user session is opened (can replace kernel/inherited values),
  • systemd, especially to the processes it manages,
  • theprocess itself (can replace PAM & kernel/inherited values, seesetrlimit(2)).

Normal users' processes cannot rise hard limits.

The Linux kernel

At boot time, Linux sets default limits to theinit (orsystemd) process, which are then inherited by all the other (children) processes. To see these limits:cat /proc/1/limits.

For example, the kernel default formaximum number of file descriptors (ulimit -n) was 1024/1024 (soft, hard), andhas been raised to 1024/4096 in Linux 2.6.39.

The defaultmaximum number of processes you're talking aboutis limited to approximately:

Total RAM in kB / 128

for x86 architectures (at least), but distributions sometimes change default kernel values, socheck your kernel source code forkernel/fork.c,fork_init(). The "number of processes" limit is called RLIMIT_NPROC there.

PAM

Usually, to ensure user authentication at login, PAM is used along with some modules (see/etc/pam.d/login).

On Debian, the PAM module responsible for setting limits is here :/lib/security/pam_limits.so.

This library will read its configuration from/etc/security/limits.conf and/etc/security/limits.d/*.conf, but even if those files are empty,pam_limits.so might use hardcoded values that you can check within the source code.

For example, on Debian, the librarywas (in the past) patched so that by default, themaximum number of processes (nproc) was unlimited, and themaximum number of files (nofile) was 1024/1024:

  case RLIMIT_NOFILE:      pl->limits[i].limit.rlim_cur = 1024;      pl->limits[i].limit.rlim_max = 1024;

Now it is apparentlypatched to cap this limit in some cases (thanksAlex O for this info).

So,check your CentOS' PAM module source code (look for RLIMIT_NPROC).

However, please note that many processes will not go through PAM (usually, if they are not launched by a logged in user, like daemons and maybe cron jobs).

systemd

Nowadays,systemd is widely used, it replacesinit and can also configure specific limits values, especially to the processes/daemons it manages and creates itself.

Some limits it uses by default can be manually configured in/etc/systemd/system.conf. There is more information available in the documentation.

answeredMar 6, 2013 at 15:38
Totor's user avatar
8
  • True, point taken, comment removed. I guess I would say that for most users, PAM is probably enabled, so I would recommend checking your /etc/security/limits.conf and /etc/security/limits.d/* files first. In this particular instance, which I also ran into, there is a 1024 process/total user threads limit imposed by default in CentOS 6 via a limits.d file.CommentedSep 12, 2014 at 17:15
  • @rogerdpack yes, PAM is certainly enabled, but, again, as I said in my answer: "please note that many processes will not go through PAM (usually, if they are not launched by a logged in user, like daemons and maybe cron jobs)". Our discussion has no added-value, therefore, if you delete all your comments, I will delete mine. Thank you.CommentedSep 15, 2014 at 8:12
  • SuSE distributions haveulimit package that provided/etc/initscript -- "a convenient place to adjust per process limits", configurable via/etc/sysconfig/ulimit.CommentedMay 7, 2016 at 8:55
  • also, Linux-PAM library reads limits set by kernel (i.e./proc/1/limits) since version 1.1.4 (released 2011).CommentedMay 7, 2016 at 9:30
  • 1
    @AlexO my answer is quite old and the Debian patch, that is not used any more, hard limited to 1024. Now indeed, is most recent versions, there is this capping. I updated my answer to mention this. Thank you.CommentedJul 12, 2024 at 15:10
15

On RHEL6 (CentOS6) "max user processes" is set to 1024 by default.
You can change this value in file:

/etc/security/limits.d/90-nproc.conf

Seehttps://bugzilla.redhat.com/show_bug.cgi?id=432903 if you'd like to complain about it :)

voretaq7's user avatar
voretaq7
80.8k18 gold badges135 silver badges222 bronze badges
answeredOct 4, 2012 at 8:46
Tomas's user avatar
2
  • I doubt this 1024 value for nproc is correct and the author said that its limits.d dir was empty, so the default value is obviously not defined there.CommentedMar 11, 2013 at 15:33
  • Totor can't argue with you technically but Tom I found it helpful so thanks!CommentedOct 11, 2013 at 17:23
7

Info on this is terrible on the internet, heres a limits.conf file i made for debian linux, showing all possible options and their maximum "safe" limits, tweak accordingly.

These are the highest values you can set, some things are hashed out, activating those causes you to error out and be unable to login to your console, modify the commented out options at your own risk, but you shouldnt need to (default is unlimited on most)

I hope this is usefull to someone, as i could not find this info anywhere, theres 4 hours of research on this file.

==== FILE START =====# /etc/security/limits.conf# #Each line describes a limit for a user in the form:##<domain>        <type>  <item>  <value>##Where:#<domain> can be:#- a user name#- a group name, with @group syntax#- the wildcard     *, for default entry#- the wildcard %, can be also used with %group syntax,#         for maxlogin limit#- NOTE: group and wildcard limits are not applied to     root.#  To apply a limit to the     root user, <domain> must be#  the literal username     root.##<type> can have the two values:#- "soft" for enforcing the soft limits#- "hard" for enforcing hard limits##<item> can be one of the following:#- core - limits the core file size (KB)#- data - max data size (KB)#- fsize - maximum filesize (KB)#- memlock - max locked-in-memory address space (KB)#- nofile - max number of open files#- rss - max resident set size (KB)#- stack - max stack size (KB)#- cpu - max CPU time (MIN)#- nproc - max number of processes#- as - address space limit (KB)#- maxlogins - max number of logins for this user#- maxsyslogins - max number of logins on the system#- priority - the priority to run user process with#- locks - max number of file locks the user can hold#- sigpending - max number of pending signals#- msgqueue - max memory used by POSIX message queues (bytes)#- nice - max nice priority allowed to raise to values: [-20, 19]#- rtprio - max realtime priority#- chroot - change     root to directory (Debian-specific)##<domain>      <type>  <item>         <value>##*               soft    core            0#root            hard    core            100000#*               hard    rss             10000#@student        hard    nproc           20#@faculty        soft    nproc           20#@faculty        hard    nproc           50#ftp             hard    nproc           0#ftp             -       chroot          /ftp#@student        -       maxlogins       4# -- Defaults:#(core) core file size                (blocks, -c) 0 (ulimit -Hc or -Sc)#(data) data seg size                  (bytes, -d) unlimited#(priority) scheduling priority               (-e) 0#(fsize) file size                    (blocks, -f) unlimited#(sigpending) pending signals                 (-i) 378197#(memlock) max locked memory          (kbytes, -l) 64# max memory size                     (kbytes, -m) unlimited#(nofile) open files                          (-n) 65536# pipe size                        (512 bytes, -p) 8#(msgqueue) POSIX message queues       (bytes, -q) 819200#(rtprio) real-time priority                  (-r) 0#(stack) stack size                   (kbytes, -s) 8192#(cpu) cpu time                      (seconds, -t) unlimited#(nproc) max user processes                   (-u) 378197# virtual memory                      (kbytes, -v) unlimited#(locks) file locks                           (-x) unlimited# --     root Limits:root               -    core            -1root               -    data            -1root               -    fsize           -1root               -    memlock         -1root               -    nofile          999999root               -    stack           -1root               -    cpu             -1root               -    nproc           -1root               -    priority        0root               -    locks           -1root               -    sigpending      -1root               -    msgqueue        -1root               -    rtprio          -1root               -    maxlogins       -1root               -    maxsyslogins    -1#root               -    rss             -1#root               -    as              -1#root               -    nice            0#root               -    chroot          -1#All Users:# -- Hard Limits*               hard    core            -1*               hard    data            -1*               hard    fsize           -1*               hard    memlock         -1*               hard    nofile          999999*               hard    stack           -1*               hard    cpu             -1*               hard    nproc           -1*               hard    priority        0*               hard    locks           -1*               hard    sigpending      -1*               hard    msgqueue        -1*               hard    rtprio          -1*               hard    maxlogins       -1*               hard    maxsyslogins    -1#*               hard    rss             -1#*               hard    as              -1#*               hard    nice            0#*               hard    chroot          -1# -- Soft Limits*               soft    core            -1*               soft    data            -1*               soft    fsize           -1*               soft    memlock         -1*               soft    nofile          999999*               soft    stack           -1*               soft    cpu             -1*               soft    nproc           -1*               soft    priority        0*               soft    locks           -1*               soft    sigpending      -1*               soft    msgqueue        -1*               soft    maxlogins       -1*               soft    maxsyslogins    -1*               soft    rtprio          -1#*               soft    rss             -1#*               soft    as              -1#*               soft    nice            0#*               soft    chroot          -1#randomuser:# -- Soft Limitsrandomuser           soft    core            -1randomuser           soft    data            -1randomuser           soft    fsize           -1randomuser           soft    memlock         -1randomuser           soft    nofile          999999randomuser           soft    stack           -1randomuser           soft    cpu             -1randomuser           soft    nproc           -1randomuser           soft    priority        0randomuser           soft    locks           -1randomuser           soft    sigpending      -1randomuser           soft    msgqueue        -1randomuser           soft    maxlogins       -1randomuser           soft    maxsyslogins    -1randomuser           soft    rtprio          -1#randomuser           soft    rss             -1#randomuser           soft    as              -1#randomuser           soft    nice            0#randomuser           soft    chroot          -1# End of file
answeredJun 6, 2017 at 3:52
XionicFire's user avatar
3

When you checked the limits, were you using the root user to do so?

From thelimits.conf manpage:

NOTE: group and wildcard limits are not applied to the root user. To set a limit for the root user, this field must contain the literal username root.

Using explicit usernames would resolve the issue in this case.

answeredAug 28, 2012 at 16:46
Christopher Cashell's user avatar
2
  • Be careful, this is probably aDebian specific "feature".CommentedMar 6, 2013 at 15:54
  • Also, thelimits.conf file is empty (as thelimits.d directory).CommentedMar 11, 2013 at 2:02
3

kernel/fork.c

max_threads = mempages / (8 * THREAD_SIZE / PAGE_SIZE);

On 64 bit Thread size is 8192

 grep -i total /proc/meminfo  MemTotal:        8069352 kB

Now i get the total in kb in division by 4

 echo $((8069352/4)) 2017338

Now i got the number of pages

 echo $((8 * 8192 / 4096) 16

The final result is

echo $((2017338/16))126083

In this way you got the thread-max parameter and the default user process limit is half

init_task.signal->rlim[RLIMIT_NPROC].rlim_cur = max_threads/2;init_task.signal->rlim[RLIMIT_NPROC].rlim_max = max_threads/2;

ulimit from root

ulimit -u62932echo $((62932*2))125864 #we are near
answeredDec 20, 2013 at 0:13
c4f4t0r's user avatar
1
2

There is one more possibility that the configuration for "noproc" is not working while configuring in /etc/security/limits.conf.

There is one more file which overrides your configuration /etc/security/limits.d/90-nproc.conf.

*          soft    nproc     1024root       soft    nproc     unlimited

Here * config will override whatever you set in previous config file. So ideally you configure your setting in this file.

answeredOct 15, 2015 at 10:24
Suyash Jain's user avatar
1

It appears to be /etc/security/limits.conf

http://ss64.com/bash/limits.conf.html

answeredFeb 5, 2012 at 4:32
jamesbtate's user avatar
2
  • 1
    I mentioned that in my post already. It has no effect, nor are those values (unlimited, 77824) set there for the respective machines (that file is empty).CommentedFeb 5, 2012 at 4:36
  • oh i saw you checked the .bashrc etc. but didn't see you mentioned this one too.CommentedFeb 5, 2012 at 5:00
0

I solved this after I struggled with this problem for more than an hour!I deleted the config and recreated the file like this:

vi /etc/security/limits.confroot    -   nproc   500000root    -   nofile  500000myuser  -   nproc   130000myuser  -   nofile  130000

After logout and login, limit was working:

# ulimit -n500000
answeredFeb 23, 2022 at 22:28
Arpatma's user avatar
0

Linux kernel

include/asm-generic/resource.h:

/* * boot-time rlimit defaults for the init task: */#define INIT_RLIMITS                            \{                                   \    [RLIMIT_CPU]        = {  RLIM_INFINITY,  RLIM_INFINITY },   \    [RLIMIT_FSIZE]      = {  RLIM_INFINITY,  RLIM_INFINITY },   \    [RLIMIT_DATA]       = {  RLIM_INFINITY,  RLIM_INFINITY },   \    [RLIMIT_STACK]      = {       _STK_LIM,  RLIM_INFINITY },   \    [RLIMIT_CORE]       = {              0,  RLIM_INFINITY },   \    [RLIMIT_RSS]        = {  RLIM_INFINITY,  RLIM_INFINITY },   \    [RLIMIT_NPROC]      = {              0,              0 },   \    [RLIMIT_NOFILE]     = {   INR_OPEN_CUR,   INR_OPEN_MAX },   \    [RLIMIT_MEMLOCK]    = {    MLOCK_LIMIT,    MLOCK_LIMIT },   \    [RLIMIT_AS]         = {  RLIM_INFINITY,  RLIM_INFINITY },   \    [RLIMIT_LOCKS]      = {  RLIM_INFINITY,  RLIM_INFINITY },   \    [RLIMIT_SIGPENDING] = {         0,         0 }, \    [RLIMIT_MSGQUEUE]   = {   MQ_BYTES_MAX,   MQ_BYTES_MAX },   \    [RLIMIT_NICE]       = { 0, 0 },             \    [RLIMIT_RTPRIO]     = { 0, 0 },             \    [RLIMIT_RTTIME]     = {  RLIM_INFINITY,  RLIM_INFINITY },   \}
answeredJun 7, 2024 at 17:15
Mario Klebsch's user avatar
2
  • Would be good to expand somewhat - this is just a source code snippet. It doesn't say /where/ in the distro these might be set or configured.CommentedJun 20, 2024 at 8:16
  • include/asm-generic/resource.h is part of the linux kernel source code. These are the settings, that are used, if really nobody changes them.CommentedJun 21, 2024 at 19:19

You mustlog in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.