René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[BUG] Data races","articleBody":"### Prerequisites\r\n\r\n* [X] Put an X between the brackets on this line if you have checked that your issue isn't already filed: https://github.com/search?l=\u0026q=repo%3Aetr%2Flibhttpserver\u0026type=Issues\r\n\r\n### Description\r\n\r\nThe documentation states that `All functions are guaranteed to be completely reentrant and thread-safe (unless differently specified)`. However, the `webserver` implementation does not use any protection while reading/modifying its member variables (`registered_resources`, `registered_resources_str`, `bans` and `allowances`).\r\n\r\n### Steps to Reproduce\r\n\r\nHere is a small test that show the issue when compiled using clang-16 with thread sanitizing enabled:\r\n```cpp\r\n\r\nLT_BEGIN_AUTO_TEST(basic_suite, thread_safety)\r\n    simple_resource resource;\r\n\r\n    std::atomic_bool done = false;\r\n    auto register_thread = std::thread([\u0026]() {\r\n        int i = 0;\r\n        using namespace std::chrono;\r\n        while (!done) {\r\n            ws-\u003eregister_resource(\r\n                    std::string(\"/route\") + std::to_string(++i), \u0026resource);\r\n        }\r\n    });\r\n\r\n    auto get_thread = std::thread([\u0026](){\r\n        while (!done) {\r\n            CURL *curl = curl_easy_init();\r\n            std::string s;\r\n            std::string url = \"localhost:\" PORT_STRING \"/route\" + std::to_string(\r\n                                            (int)((rand() * 10000000.0) / RAND_MAX));\r\n            curl_easy_setopt(curl, CURLOPT_URL, url.c_str());\r\n            curl_easy_setopt(curl, CURLOPT_HTTPGET, 1L);\r\n            curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writefunc);\r\n            curl_easy_setopt(curl, CURLOPT_WRITEDATA, \u0026s);\r\n            curl_easy_perform(curl);\r\n            curl_easy_cleanup(curl);\r\n        }\r\n    });\r\n\r\n    using namespace std::chrono_literals;\r\n    std::this_thread::sleep_for(10s);\r\n    done = true;\r\n    if (register_thread.joinable()) {\r\n        register_thread.join();\r\n    }\r\n    if (get_thread.joinable()) {\r\n        get_thread.join();\r\n    }\r\n    LT_CHECK_EQ(1, 1);\r\nLT_END_AUTO_TEST(thread_safety)\r\n```\r\n\r\n**Expected behavior:** \r\n\r\nNo data races are detected by clang.\r\n\r\n**Actual behavior:**\r\nClang reports the following data race (and others):\r\n\r\n```txt\r\nRunning test (1): thread_safety\r\n==================\r\nWARNING: ThreadSanitizer: data race (pid=21795)\r\n  Read of size 8 at 0x7b1400003550 by thread T1:\r\n    #0 memcmp /clang-16.0.1/projects/compiler-rt/lib/tsan/rtl/../../sanitizer_common/sanitizer_common_interceptors.inc:939:3 (lt-basic+0x67439)\r\n    #1 memcmp /clang-16.0.1/projects/compiler-rt/lib/tsan/rtl/../../sanitizer_common/sanitizer_common_interceptors.inc:935:1 (lt-basic+0x67439)\r\n    #2 httpserver::webserver::finalize_answer(MHD_Connection*, httpserver::details::modded_request*, char const*) \u003cnull\u003e (libhttpserver.so.0+0x227b2)\r\n    #3 MHD_connection_handle_idle \u003cnull\u003e (libmicrohttpd.so.12+0xd1ab) (BuildId: 72677d816e65dce550957833f9aea14ac2e0e4c8)\r\n    #4 call_handlers daemon.c (libmicrohttpd.so.12+0x118af) (BuildId: 72677d816e65dce550957833f9aea14ac2e0e4c8)\r\n    #5 MHD_epoll daemon.c (libmicrohttpd.so.12+0x194ad) (BuildId: 72677d816e65dce550957833f9aea14ac2e0e4c8)\r\n    #6 MHD_polling_thread daemon.c (libmicrohttpd.so.12+0x1a02e) (BuildId: 72677d816e65dce550957833f9aea14ac2e0e4c8)\r\n    #7 named_thread_starter mhd_threads.c (libmicrohttpd.so.12+0x24515) (BuildId: 72677d816e65dce550957833f9aea14ac2e0e4c8)\r\n\r\n  Previous write of size 8 at 0x7b1400003550 by thread T2:\r\n    #0 operator new(unsigned long) /clang-16.0.1/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:64:3 (lt-basic+0xea6e5)\r\n    #1 std::pair\u003cstd::_Rb_tree_iterator\u003cstd::pair\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e const, httpserver::http_resource*\u003e\u003e, bool\u003e std::_Rb_tree\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e, std::pair\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e const, httpserver::http_resource*\u003e, std::_Select1st\u003cstd::pair\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e const, httpserver::http_resource*\u003e\u003e, std::less\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e\u003e, std::allocator\u003cstd::pair\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e const, httpserver::http_resource*\u003e\u003e\u003e::_M_emplace_unique\u003cstd::pair\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e, httpserver::http_resource*\u003e\u003e(std::pair\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e, httpserver::http_resource*\u003e\u0026\u0026) \u003cnull\u003e (libhttpserver.so.0+0x2669a)\r\n\r\n  Location is heap block of size 72 at 0x7b1400003520 allocated by thread T2:\r\n    #0 operator new(unsigned long) /clang-16.0.1/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:64:3 (lt-basic+0xea6e5)\r\n    #1 std::pair\u003cstd::_Rb_tree_iterator\u003cstd::pair\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e const, httpserver::http_resource*\u003e\u003e, bool\u003e std::_Rb_tree\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e, std::pair\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e const, httpserver::http_resource*\u003e, std::_Select1st\u003cstd::pair\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e const, httpserver::http_resource*\u003e\u003e, std::less\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e\u003e, std::allocator\u003cstd::pair\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e const, httpserver::http_resource*\u003e\u003e\u003e::_M_emplace_unique\u003cstd::pair\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e, httpserver::http_resource*\u003e\u003e(std::pair\u003cstd::__cxx11::basic_string\u003cchar, std::char_traits\u003cchar\u003e, std::allocator\u003cchar\u003e\u003e, httpserver::http_resource*\u003e\u0026\u0026) \u003cnull\u003e (libhttpserver.so.0+0x2669a)\r\n\r\n  Thread T1 'MHD-single' (tid=21814, running) created by main thread at:\r\n    #0 pthread_create /clang-16.0.1/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1048:3 (lt-basic+0x2db2f)\r\n    #1 MHD_create_thread_ \u003cnull\u003e (libmicrohttpd.so.12+0x2443d) (BuildId: 72677d816e65dce550957833f9aea14ac2e0e4c8)\r\n    #2 MHD_create_named_thread_ \u003cnull\u003e (libmicrohttpd.so.12+0x24602) (BuildId: 72677d816e65dce550957833f9aea14ac2e0e4c8)\r\n    #3 MHD_start_daemon_va \u003cnull\u003e (libmicrohttpd.so.12+0x1e4de) (BuildId: 72677d816e65dce550957833f9aea14ac2e0e4c8)\r\n    #4 MHD_start_daemon \u003cnull\u003e (libmicrohttpd.so.12+0x1a1d6) (BuildId: 72677d816e65dce550957833f9aea14ac2e0e4c8)\r\n    #5 httpserver::webserver::start(bool) \u003cnull\u003e (libhttpserver.so.0+0x20090)\r\n\r\n  Thread T2 (tid=21815, running) created by main thread at:\r\n    #0 pthread_create /clang-16.0.1/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1048:3 (lt-basic+0x2db2f)\r\n    #1 std::thread::_M_start_thread(std::unique_ptr\u003cstd::thread::_State, std::default_delete\u003cstd::thread::_State\u003e\u003e, void (*)()) \u003cnull\u003e (libstdc++.so.6+0xed8f9) (BuildId: 06e9553aa6e15b32e77410de83bbdd7d208a620d)\r\n\r\nSUMMARY: ThreadSanitizer: data race (/home/florian/work/libhttpserver/build/src/.libs/libhttpserver.so.0+0x227b2) in httpserver::webserver::finalize_answer(MHD_Connection*, httpserver::details::modded_request*, char const*)\r\n==================\r\n- Time spent during \"thread_safety\": 10702.4 ms\r\n==================\r\n```\r\n\r\n\r\n**Reproduces how often:** 100%\r\n\r\n### Versions\r\n\r\n* OS version : Linux 5f8a17221539 5.19.0-46-generic #47~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Jun 21 15:35:31 UTC 2 x86_64 GNU/Linux\r\n* libhttpserver version: master (d249ba682441dbb979146482aff01a7073ed165a) compiled locally\r\n* libmicrohttpd version: 0.9.64 compiled locally\r\n","author":{"url":"https://github.com/FlorianChevassu","@type":"Person","name":"FlorianChevassu"},"datePublished":"2023-07-24T15:26:14.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":5},"url":"https://github.com/329/libhttpserver/issues/329"}