{"id":"CVE-2024-53054","details":"In the Linux kernel, the following vulnerability has been resolved:\n\ncgroup/bpf: use a dedicated workqueue for cgroup bpf destruction\n\nA hung_task problem shown below was found:\n\nINFO: task kworker/0:0:8 blocked for more than 327 seconds.\n\"echo 0 \u003e /proc/sys/kernel/hung_task_timeout_secs\" disables this message.\nWorkqueue: events cgroup_bpf_release\nCall Trace:\n \u003cTASK\u003e\n __schedule+0x5a2/0x2050\n ? find_held_lock+0x33/0x100\n ? wq_worker_sleeping+0x9e/0xe0\n schedule+0x9f/0x180\n schedule_preempt_disabled+0x25/0x50\n __mutex_lock+0x512/0x740\n ? cgroup_bpf_release+0x1e/0x4d0\n ? cgroup_bpf_release+0xcf/0x4d0\n ? process_scheduled_works+0x161/0x8a0\n ? cgroup_bpf_release+0x1e/0x4d0\n ? mutex_lock_nested+0x2b/0x40\n ? __pfx_delay_tsc+0x10/0x10\n mutex_lock_nested+0x2b/0x40\n cgroup_bpf_release+0xcf/0x4d0\n ? process_scheduled_works+0x161/0x8a0\n ? trace_event_raw_event_workqueue_execute_start+0x64/0xd0\n ? process_scheduled_works+0x161/0x8a0\n process_scheduled_works+0x23a/0x8a0\n worker_thread+0x231/0x5b0\n ? __pfx_worker_thread+0x10/0x10\n kthread+0x14d/0x1c0\n ? __pfx_kthread+0x10/0x10\n ret_from_fork+0x59/0x70\n ? __pfx_kthread+0x10/0x10\n ret_from_fork_asm+0x1b/0x30\n \u003c/TASK\u003e\n\nThis issue can be reproduced by the following pressuse test:\n1. A large number of cpuset cgroups are deleted.\n2. Set cpu on and off repeatly.\n3. Set watchdog_thresh repeatly.\nThe scripts can be obtained at LINK mentioned above the signature.\n\nThe reason for this issue is cgroup_mutex and cpu_hotplug_lock are\nacquired in different tasks, which may lead to deadlock.\nIt can lead to a deadlock through the following steps:\n1. A large number of cpusets are deleted asynchronously, which puts a\n   large number of cgroup_bpf_release works into system_wq. The max_active\n   of system_wq is WQ_DFL_ACTIVE(256). Consequently, all active works are\n   cgroup_bpf_release works, and many cgroup_bpf_release works will be put\n   into inactive queue. As illustrated in the diagram, there are 256 (in\n   the acvtive queue) + n (in the inactive queue) works.\n2. Setting watchdog_thresh will hold cpu_hotplug_lock.read and put\n   smp_call_on_cpu work into system_wq. However step 1 has already filled\n   system_wq, 'sscs.work' is put into inactive queue. 'sscs.work' has\n   to wait until the works that were put into the inacvtive queue earlier\n   have executed (n cgroup_bpf_release), so it will be blocked for a while.\n3. Cpu offline requires cpu_hotplug_lock.write, which is blocked by step 2.\n4. Cpusets that were deleted at step 1 put cgroup_release works into\n   cgroup_destroy_wq. They are competing to get cgroup_mutex all the time.\n   When cgroup_metux is acqured by work at css_killed_work_fn, it will\n   call cpuset_css_offline, which needs to acqure cpu_hotplug_lock.read.\n   However, cpuset_css_offline will be blocked for step 3.\n5. At this moment, there are 256 works in active queue that are\n   cgroup_bpf_release, they are attempting to acquire cgroup_mutex, and as\n   a result, all of them are blocked. Consequently, sscs.work can not be\n   executed. Ultimately, this situation leads to four processes being\n   blocked, forming a deadlock.\n\nsystem_wq(step1)\t\tWatchDog(step2)\t\t\tcpu offline(step3)\tcgroup_destroy_wq(step4)\n...\n2000+ cgroups deleted asyn\n256 actives + n inactives\n\t\t\t\t__lockup_detector_reconfigure\n\t\t\t\tP(cpu_hotplug_lock.read)\n\t\t\t\tput sscs.work into system_wq\n256 + n + 1(sscs.work)\nsscs.work wait to be executed\n\t\t\t\twarting sscs.work finish\n\t\t\t\t\t\t\t\tpercpu_down_write\n\t\t\t\t\t\t\t\tP(cpu_hotplug_lock.write)\n\t\t\t\t\t\t\t\t...blocking...\n\t\t\t\t\t\t\t\t\t\t\tcss_killed_work_fn\n\t\t\t\t\t\t\t\t\t\t\tP(cgroup_mutex)\n\t\t\t\t\t\t\t\t\t\t\tcpuset_css_offline\n\t\t\t\t\t\t\t\t\t\t\tP(cpu_hotplug_lock.read)\n\t\t\t\t\t\t\t\t\t\t\t...blocking...\n256 cgroup_bpf_release\nmutex_lock(&cgroup_mutex);\n..blocking...\n\nTo fix the problem, place cgroup_bpf_release works on a dedicated\nworkqueue which can break the loop and solve the problem. System wqs are\nfor misc things which shouldn't create a large number of concurrent work\nitems. If something is going to generate \u003e\n---truncated---","modified":"2024-11-24T17:48:31.959249Z","published":"2024-11-19T18:15:25Z","withdrawn":"2024-11-28T21:43:32.810921Z","references":[{"type":"WEB","url":"https://git.kernel.org/stable/c/0d86cd70fc6a7ba18becb52ad8334d5ad3eca530"},{"type":"WEB","url":"https://git.kernel.org/stable/c/117932eea99b729ee5d12783601a4f7f5fd58a23"},{"type":"WEB","url":"https://git.kernel.org/stable/c/6dab3331523ba73db1345d19e6f586dcd5f6efb4"},{"type":"WEB","url":"https://git.kernel.org/stable/c/71f14a9f5c7db72fdbc56e667d4ed42a1a760494"},{"type":"ADVISORY","url":"https://security-tracker.debian.org/tracker/CVE-2024-53054"}],"affected":[{"package":{"name":"linux","ecosystem":"Debian:11","purl":"pkg:deb/debian/linux?arch=source"},"ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"}]}],"versions":["5.10.103-1","5.10.103-1~bpo10+1","5.10.106-1","5.10.113-1","5.10.120-1","5.10.120-1~bpo10+1","5.10.127-1","5.10.127-2","5.10.127-2~bpo10+1","5.10.136-1","5.10.140-1","5.10.148-1","5.10.149-1","5.10.149-2","5.10.158-1","5.10.158-2","5.10.162-1","5.10.178-1","5.10.178-2","5.10.178-3","5.10.179-1","5.10.179-2","5.10.179-3","5.10.179-4","5.10.179-5","5.10.191-1","5.10.197-1","5.10.205-1","5.10.205-2","5.10.209-1","5.10.209-2","5.10.216-1","5.10.218-1","5.10.221-1","5.10.223-1","5.10.226-1","5.10.46-4","5.10.46-5","5.10.70-1","5.10.70-1~bpo10+1","5.10.84-1","5.10.92-1","5.10.92-1~bpo10+1","5.10.92-2","5.13.12-1~exp1","5.13.9-1~exp1","5.13.9-1~exp2","5.14-1~exp1","5.14-1~exp2","5.14.1-1~exp1","5.14.12-1","5.14.16-1","5.14.2-1~exp1","5.14.3-1~exp1","5.14.6-1","5.14.6-2","5.14.6-3","5.14.9-1","5.14.9-2","5.14.9-2~bpo11+1","5.15-1~exp1","5.15.1-1~exp1","5.15.15-1","5.15.15-2","5.15.15-2~bpo11+1","5.15.2-1~exp1","5.15.3-1","5.15.5-1","5.15.5-2","5.15.5-2~bpo11+1","5.16.10-1","5.16.11-1","5.16.11-1~bpo11+1","5.16.12-1","5.16.12-1~bpo11+1","5.16.14-1","5.16.18-1","5.16.3-1~exp1","5.16.4-1~exp1","5.16.7-1","5.16.7-2","5.16~rc1-1~exp1","5.16~rc3-1~exp1","5.16~rc4-1~exp1","5.16~rc5-1~exp1","5.16~rc6-1~exp1","5.16~rc7-1~exp1","5.16~rc8-1~exp1","5.17.1-1~exp1","5.17.11-1","5.17.3-1","5.17.6-1","5.17~rc3-1~exp1","5.17~rc4-1~exp1","5.17~rc5-1~exp1","5.17~rc6-1~exp1","5.17~rc7-1~exp1","5.17~rc8-1~exp1","5.18-1~exp1","5.18.14-1","5.18.14-1~bpo11+1","5.18.16-1","5.18.16-1~bpo11+1","5.18.2-1","5.18.2-1~bpo11+1","5.18.5-1","5.19-1~exp1","5.19.11-1","5.19.11-1~bpo11+1","5.19.6-1","5.19~rc4-1~exp1","5.19~rc6-1~exp1","6.0-1~exp1","6.0.10-1","6.0.10-2","6.0.12-1","6.0.12-1+alpha","6.0.12-1~bpo11+1","6.0.2-1","6.0.3-1","6.0.3-1~bpo11+1","6.0.5-1","6.0.6-1","6.0.6-2","6.0.7-1","6.0.8-1","6.0~rc7-1~exp1","6.1.1-1~exp1","6.1.1-1~exp2","6.1.106-1","6.1.106-2","6.1.106-3","6.1.11-1","6.1.112-1","6.1.115-1","6.1.12-1","6.1.12-1~bpo11+1","6.1.15-1","6.1.15-1~bpo11+1","6.1.2-1~exp1","6.1.20-1","6.1.20-1~bpo11+1","6.1.20-2","6.1.20-2~bpo11+1","6.1.25-1","6.1.27-1","6.1.27-1~bpo11+1","6.1.37-1","6.1.38-1","6.1.38-2","6.1.38-2~bpo11+1","6.1.38-3","6.1.38-4","6.1.38-4~bpo11+1","6.1.4-1","6.1.52-1","6.1.55-1","6.1.55-1~bpo11+1","6.1.64-1","6.1.66-1","6.1.67-1","6.1.69-1","6.1.69-1~bpo11+1","6.1.7-1","6.1.76-1","6.1.76-1~bpo11+1","6.1.8-1","6.1.8-1+sh4","6.1.82-1","6.1.85-1","6.1.90-1","6.1.90-1~bpo11+1","6.1.94-1","6.1.94-1~bpo11+1","6.1.98-1","6.1.99-1","6.10-1~exp1","6.10.1-1~exp1","6.10.11-1","6.10.11-1~bpo12+1","6.10.12-1","6.10.3-1","6.10.4-1","6.10.6-1","6.10.6-1~bpo12+1","6.10.7-1","6.10.9-1","6.11-1~exp1","6.11.10-1","6.11.2-1","6.11.4-1","6.11.5-1","6.11.5-1~bpo12+1","6.11.6-1","6.11.7-1","6.11.9-1","6.11~rc4-1~exp1","6.11~rc5-1~exp1","6.12~rc6-1~exp1","6.1~rc3-1~exp1","6.1~rc5-1~exp1","6.1~rc6-1~exp1","6.1~rc7-1~exp1","6.1~rc8-1~exp1","6.3.1-1~exp1","6.3.11-1","6.3.2-1~exp1","6.3.4-1~exp1","6.3.5-1~exp1","6.3.7-1","6.3.7-1~bpo12+1","6.4.1-1~exp1","6.4.11-1","6.4.13-1","6.4.4-1","6.4.4-1~bpo12+1","6.4.4-2","6.4.4-3","6.4.4-3~bpo12+1","6.4~rc6-1~exp1","6.4~rc7-1~exp1","6.5.1-1~exp1","6.5.10-1","6.5.10-1~bpo12+1","6.5.13-1","6.5.3-1","6.5.3-1~bpo12+1","6.5.6-1","6.5.8-1","6.5~rc4-1~exp1","6.5~rc6-1~exp1","6.5~rc7-1~exp1","6.6.11-1","6.6.13-1","6.6.13-1~bpo12+1","6.6.15-1","6.6.15-2","6.6.3-1~exp1","6.6.4-1~exp1","6.6.7-1~exp1","6.6.8-1","6.6.9-1","6.7-1~exp1","6.7.1-1~exp1","6.7.12-1","6.7.12-1~bpo12+1","6.7.4-1~exp1","6.7.7-1","6.7.9-1","6.7.9-2","6.8.11-1","6.8.12-1","6.8.12-1~bpo12+1","6.8.9-1","6.9.10-1","6.9.10-1~bpo12+1","6.9.11-1","6.9.12-1","6.9.2-1~exp1","6.9.7-1","6.9.7-1~bpo12+1","6.9.8-1","6.9.9-1"],"ecosystem_specific":{"urgency":"not yet assigned"},"database_specific":{"source":"https://storage.googleapis.com/cve-osv-conversion/osv-output/CVE-2024-53054.json"}},{"package":{"name":"linux","ecosystem":"Debian:12","purl":"pkg:deb/debian/linux?arch=source"},"ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"6.1.119-1"}]}],"versions":["6.1.106-1","6.1.106-2","6.1.106-3","6.1.112-1","6.1.115-1","6.1.27-1","6.1.37-1","6.1.38-1","6.1.38-2","6.1.38-2~bpo11+1","6.1.38-3","6.1.38-4","6.1.38-4~bpo11+1","6.1.52-1","6.1.55-1","6.1.55-1~bpo11+1","6.1.64-1","6.1.66-1","6.1.67-1","6.1.69-1","6.1.69-1~bpo11+1","6.1.76-1","6.1.76-1~bpo11+1","6.1.82-1","6.1.85-1","6.1.90-1","6.1.90-1~bpo11+1","6.1.94-1","6.1.94-1~bpo11+1","6.1.98-1","6.1.99-1"],"ecosystem_specific":{"urgency":"not yet assigned"},"database_specific":{"source":"https://storage.googleapis.com/cve-osv-conversion/osv-output/CVE-2024-53054.json"}},{"package":{"name":"linux","ecosystem":"Debian:13","purl":"pkg:deb/debian/linux?arch=source"},"ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"6.11.7-1"}]}],"versions":["6.1.106-1","6.1.106-2","6.1.106-3","6.1.112-1","6.1.115-1","6.1.27-1","6.1.37-1","6.1.38-1","6.1.38-2","6.1.38-2~bpo11+1","6.1.38-3","6.1.38-4","6.1.38-4~bpo11+1","6.1.52-1","6.1.55-1","6.1.55-1~bpo11+1","6.1.64-1","6.1.66-1","6.1.67-1","6.1.69-1","6.1.69-1~bpo11+1","6.1.76-1","6.1.76-1~bpo11+1","6.1.82-1","6.1.85-1","6.1.90-1","6.1.90-1~bpo11+1","6.1.94-1","6.1.94-1~bpo11+1","6.1.98-1","6.1.99-1","6.10-1~exp1","6.10.1-1~exp1","6.10.11-1","6.10.11-1~bpo12+1","6.10.12-1","6.10.3-1","6.10.4-1","6.10.6-1","6.10.6-1~bpo12+1","6.10.7-1","6.10.9-1","6.11-1~exp1","6.11.2-1","6.11.4-1","6.11.5-1","6.11.5-1~bpo12+1","6.11.6-1","6.11~rc4-1~exp1","6.11~rc5-1~exp1","6.3.1-1~exp1","6.3.11-1","6.3.2-1~exp1","6.3.4-1~exp1","6.3.5-1~exp1","6.3.7-1","6.3.7-1~bpo12+1","6.4.1-1~exp1","6.4.11-1","6.4.13-1","6.4.4-1","6.4.4-1~bpo12+1","6.4.4-2","6.4.4-3","6.4.4-3~bpo12+1","6.4~rc6-1~exp1","6.4~rc7-1~exp1","6.5.1-1~exp1","6.5.10-1","6.5.10-1~bpo12+1","6.5.13-1","6.5.3-1","6.5.3-1~bpo12+1","6.5.6-1","6.5.8-1","6.5~rc4-1~exp1","6.5~rc6-1~exp1","6.5~rc7-1~exp1","6.6.11-1","6.6.13-1","6.6.13-1~bpo12+1","6.6.15-1","6.6.15-2","6.6.3-1~exp1","6.6.4-1~exp1","6.6.7-1~exp1","6.6.8-1","6.6.9-1","6.7-1~exp1","6.7.1-1~exp1","6.7.12-1","6.7.12-1~bpo12+1","6.7.4-1~exp1","6.7.7-1","6.7.9-1","6.7.9-2","6.8.11-1","6.8.12-1","6.8.12-1~bpo12+1","6.8.9-1","6.9.10-1","6.9.10-1~bpo12+1","6.9.11-1","6.9.12-1","6.9.2-1~exp1","6.9.7-1","6.9.7-1~bpo12+1","6.9.8-1","6.9.9-1"],"ecosystem_specific":{"urgency":"not yet assigned"},"database_specific":{"source":"https://storage.googleapis.com/cve-osv-conversion/osv-output/CVE-2024-53054.json"}}],"schema_version":"1.7.3"}