diff --git a/doc/rtd/content/02_basic_functions/mlpro_bf/sub/layer1_computation/02_multitasking.rst b/doc/rtd/content/02_basic_functions/mlpro_bf/sub/layer1_computation/02_multitasking.rst index ec7f918a7..2e171e3e6 100644 --- a/doc/rtd/content/02_basic_functions/mlpro_bf/sub/layer1_computation/02_multitasking.rst +++ b/doc/rtd/content/02_basic_functions/mlpro_bf/sub/layer1_computation/02_multitasking.rst @@ -3,20 +3,17 @@ Multitasking ============ -In MLPro, the two essential techniques **Multithreading** and **Multiprocessing** for the asynchronous execution of -program parts are summarized under multitasking. Both techniques allow the use of several or even all cores of the -CPU(s), which can significantly increase the execution speed of the respective program. The basic prerequisite for -this is, of course, that significant parts of the program can actually be executed in parallel. - -While with multithreading program parts are executed in parallel within the same process, with multiprocessing they -are started in separate processes. The latter uses computing resources more effectively but requires more -administrative effort on the part of the operating system. Therefore, it is hard to say in advance which of the two -techniques is the better choice for a specific implementation. What can be said, however, is that multithreading is -the more straightforward technique to use with good acceleration of parallel programs. Multiprocessing requires a -little more detailed knowledge of the internal mechanisms of the runtime environment. It unfolds its strengths, in -particular with long-running parallel program parts, since the administrative overhead is not so significant here. -The pros and cons of multithreading and multiprocessing can be excellently discussed. For this purpose, reference is -made to relevant sources on the Internet. +In MLPro, **multithreading** and **multiprocessing** support asynchronous execution of +independent program parts. Multithreading runs work in threads within one process, +while multiprocessing uses separate processes and requires explicit interprocess +communication for shared state. + +Both approaches can improve throughput, but their performance depends on the workload +and synchronization overhead. In conventional CPython builds, the Global Interpreter +Lock (GIL) limits simultaneous execution of Python bytecode by threads; native extensions +may release it. Free-threaded Python builds have different concurrency characteristics. +Multiprocessing can execute CPU-bound Python code across multiple cores, at the cost of +additional process and data-transfer overhead. The details of the multitasking implemented in MLPro are explained in more detail below: @@ -32,12 +29,24 @@ different degrees: **Asynchronous execution of methods** -In order to enable the asynchronous execution of methods of a class, MLPro provides the property class **Async**. -In particular, this includes the method **_start_async()**, which allows the execution of another method in a separate -thread or process. +MLPro provides the property class **Async** for asynchronous execution of callables. +Its ``_start_async()`` method executes the target synchronously or starts a new thread +or process, depending on the configured maximum range and the optional per-call range. +``wait_async_tasks()`` joins the asynchronous operations registered on the current +``Async`` instance. +``Async`` derives from ``Range`` and does not inherit from ``Log``. Its constructor +accepts ``p_range_max``, the optional shared-object class ``p_class_shared``, and +additional keyword arguments for shared-object construction. An associated shared +object can be accessed using ``get_so()`` or assigned using ``assign_so()``. -**Tasks und workflows** +Each asynchronous call creates a new thread or process; ``Async`` does not provide a +bounded worker pool. Its task list is not synchronized for concurrent starts and waits +on the same instance. In addition, exceptions raised by worker threads are not +forwarded by ``wait_async_tasks()``. Applications must account for these limitations +when orchestrating nested or concurrent work. + +**Tasks and workflows** A fundamental and consistently used concept in MLPro is that of tasks and workflows. A task is a class that can be executed in one of the three possible ranges mentioned above. Tasks can in turn be grouped into workflows. diff --git a/doc/rtd/content/99_appendices/appendix2/sub/mlpro_bf/layer1_computation/images/.$MLPro-BF-MT_class_diagram.drawio.bkp b/doc/rtd/content/99_appendices/appendix2/sub/mlpro_bf/layer1_computation/images/.$MLPro-BF-MT_class_diagram.drawio.bkp deleted file mode 100644 index 9c6837c9c..000000000 --- a/doc/rtd/content/99_appendices/appendix2/sub/mlpro_bf/layer1_computation/images/.$MLPro-BF-MT_class_diagram.drawio.bkp +++ /dev/null @@ -1 +0,0 @@ -7V1pd6JKE/41OTf3PSc5bKJ+dM1mTGLWuV88CC0wAm1Yos6vf7tZFOzWkIDoRDL3JtIgvTxV1VXVVd0nfMucX9jSVLuFCjBOOEaZn/DtE47j6hUB/cEli6CkUqtWghLV1pWgjF0VPOp/QFjIhKWergAn8aALoeHq02ShDC0LyG6iTLJtOEs+NoZGstappAKi4FGWDLL0VVdcLSwVo47hG5dAV7Wwao7nxeCOKUVPh11xNEmBs1gR3znhWzaEbvDJnLeAgYcvGpjXq8Wr0ZuIF9cPzrv03Lx56r+cBS/rfuUryz7YwHK//erLS/tDM50HZ9FR1Vup8XzRl84imD4kwwtHLOysu4iGULWhN03ZhLCpH8B2wZwGsDSKXrsaQkR9AJrAtRfoufBbQtiwkO7OBJEPS2YrGIVaWKbFEYzwkkLSUZcvX40O+hAO0FcGi6UMlmigeptjiIYFd9KAtn9HfPcwXTRPOJ7xf+JFjBPxSVTGxm+LavjXf/UogUb0FK7wLHhNAz3ACtM5+YoeUIGl+E+E77LX344GYkQpC/qz1hDiy2lKPhuZ8Tg5Mp/1VKR1dGv/aH1xppJFrS1sJ67JVkenCDb0HyIXJvbp380NCHv7aS+ocKG/kjlFH6yRMw2Ggyza1K3EY1e4c1MbqjZwnFSvzbXIrx5xia5Irg6tghswAJKy8OcZ9MtzQGzMAti/RqBJ1t2E1Bfp72tckQcfJkoSjZdc19ZHngv8seziqQ+4GlRO/80TpT6YYZr0RoYuY+G8qrQbVBchNsLt0M2pAUw01wDly1IpT0C/CR+tiq3DHBNKy5rQu+rJpqw9kA82PjBd3dKArfvD/RlImoT7rjvrMKFZ2AabYcmXExJdGBJjmyuatCYMv8YjIfXb0EU6kD9Y0Xh2420/HAYgh5T5TpeTdBXr/u2eyGlD3/MSrGkbu6ZiI2UZl2uuiRTSNoulhaGrFvpsgDG+gxVqHVk1jbDYhXiUZxoa2cepJOPXzJAFh8qQsm4pAKuxTDgcreVYoBEQRQaXO64NJyBxxx8bdCeycnArxrphxB4a+z/LDsStge3Wxbo5QKr9e9TqOUKrb4Mp0psxRWLlZQ0rdKeBDVN0NTKgPEniZkgjYNxDR/cVH75tB11YAthbu2/qioLf/Bngy+dQ9V3diKrD+IZGN+rHl4EBSsJ63miNnbGV+ppBxrERTDHoKhTkKluACyu8h7rPtWFt9fN6vS4u/1UTFbMcc84wPMNWalX0W2TY5Osd6NkyCN8YN4fXKqkI5zWhxkb/qvyXanElWwXu57XUa8nvwfHYAYmvIFqSFrEnpvgFzubB4Zn1hq5xQPDCFT8s0czAIjzBIleBXMc8Mrahebhcwhwnl/DcOV8TRIZnq/i3sBMm2V7JV3lkERvjbRyTG1ULn/u+nJluGpIFIiIK7/AxUpSxvmJvnKNlTTeUnrSAHkbacSV5El01NWjrf9BrpYgv0G07olRkAMSfeMTfDOkZWfXomfuIftllUU9y3PAZpAEZ0tTRA38bfsREeOhWE7ou4tfwRQllITH1BxpC5DOlKQM1GcgyTZcY1SpCBb9QtSVFBwkNZKlCRCPbLUzFiO4mWVQg+VOg8Oc6cVP4c4Ddm5ZqgFVdhLSm1IYmF4q6IyarkwxEY5bkgiZGzNkJO1Q+ZwdD91khJslJMkkropExK+uW+oS5pH3Grkp6/hd9HgtLBuHA8D7JulLkRWa2TCXhPIqaXWmeVNp+iY0o0UItlnSfpgDilhlw3C9Qm5Ca2hZJID8hr9pm6sqm3IqljMsi48Zjji7jFHEkVsSDlXGlkNswlNVSyH1KbmJqcjsQIVcrhVwGIadIoDamCjlRroHR+FCFXKnJbRrKeinkPiW3WmpyK1zI1XvzxsvlaPpxa1nz5iW8/qPby5kr7mNRVBAJMjQiGlShJRmdVemaWFg904M+Vhiq38B1F6GkkjwXJoki7j9WwFjyfHc2xbmTdLiAue6+4c/nlfDqV+xOex6/WIQXJHDbIkwCD0b4XNt7a94Z3V+VD8i9vc9erlrqWYhD4ITYMq5pfTw2MCRX/0iGFW2UIV/26olrjiOB59fCeda+UWM++UZmV+C2YY2JlkcNQaaUE+6GCZf/3oRrSI4TvfyTCZYyOXf9ny8z1RfmXoL6uDo5IfIcbf6t/qQJmMoklKi6IZaTQzj6jYqDEKAeFpwcc2p6OBrShjJwHDRDoiKEIl4XtQHC1VL/JVgrXC1MtUSYnIkFcibGRRB9d2z4BKyhSR1YlNk5OQs3cSwUlu5oPuZa6JpdXX9jit4qarLM0FQKXKeI78zRtiqMpPGU/e/ptyjePbX7plA9Ix3KPuxotIeu5ExKIFMCWeH2DCSFfyUZz/4+jk7Iwy3upIHDAg3dcUtsU2JbrRWH7duTpjE3gL1+lUa9oTi4epvf07BFKgASwhGsHKPocgloWkBZRigOUfp0S4vLPl57NydYKfH2VFh3ZvCSpsZwqFu6Oxye4h4PbaQh+tFp+B4zwFfnreGg0b/oDO8Hd63O42Mgon+eBkU3y3dHCmlldg60QOdwjqAFrFgFdOD6X+1DxOB+fD4uMYFvJzYQJpIb3kVkwJytQvCYEYRGSRnZKEMsUOWmUwYZmuRZPm2UXJ8R23qBWji1yRUyIkHWALanVozPldI9q9XMFqiR03EmgxR8nJEIL4HOEWihQEWdanqRerqkKKH1FUM6mMSD4hL37LhXU2ryO8Od1N5U4JK4l0hnRbpeoDpGRZrUxlZIO6VKlhFfPnJs7A1f0sMtG0CyS4TzQljYuzJG+s4KiDPYGlHgPxBFFJzE4gnO6wJ3si2mAF/cA1tHQ4MXmncRZ8CFIi8eaLCNFvILNMjmXavsA+ZvhpPkDFkU4Fh8bEg2VwdpJjWchYUTtU8HgS8Ua849SFkyLqMxMoU/Hng0hkjslpM+GiPKT/sR0RjvSkepMO0H5/l18jKtchP3iaFYH0MZAzp0gsilcBn3h+ktKTWUXIK9i9VQONKNMHTwxgnB4u2GeLQSzwMItpjKmtF8V61fE21huupTtf/QoQZb4HktEWtx3FEWX0e1yDALqtSlMKk5NSVLUpGSEWG6Fvt23pQccBs9U0L9twRgRG6JMgAjV1wLjMDYatpuj8AYmtL8ZHsUBrZL8HZWrG/MRWFzWT5MkypcVP0yCCDv2gyoqkFwblARMrJQL3t3F8NGr3dSeq5zIPaUE1akruQ+YVHUENkGyDJByuXpvz9atywa67TG6e6cnGRsGV6nKIHO2VRkC7QV65VJdSy8vUHHYPhna9Tj7j0KU6NZA6GEofanMGQ48o0I7VKMZ0ZcKNCapCstpJd06LsjA6MyXG4OPL/5awrptIlk5OqOlJY0H/73v+lwMkOD4QTEn4iPjPZtKtkhg68srVazJScuGzuQM91M0t24h6VctM0Kc6Hr8ouLaberX99MmkqfB+DtsXM1oimvQwUYyFArwc0K7t6X5HkS3XsbTlEPcDN9QzhY+gv30gUWHngs0iV8Hy/crZ4L9hj2Z0S/rUgMaDa0oIfvOd7obOxZMnaIOAThrFbZsMuFtvlsfKn3yzspJZbt1pfRLEgsXn7DYxT3BpEuIwoJ5bI8VyeW53jachlt29toCS0LTb0o4n8ft2BQ6WttbfTStGo35pKm9rf+H2463Gqd7DQCgNr7sPOfxmxE0qL4CIBtzY7JgSc/rZiJ6Y6nYUwAlgedDzRcS386Lrk3oBuI5eAS2I7uuOipgwoZEJiTwwkZ+JkbOBAhA3wtrbu5zmxxwf11IQNURiMXr1rDp1/3naXN9g9mvH9+nGpFUttWOZRah6JAvTMVij7dkfv+tYaN56e7wXN/2L/rd1aGsY/vMcSD5ANtkRmA1BYL5Ky4ghb9v4YsWyKbEtki8//oyNLEcIRs7+7ufgkqudxcgrph7aXIbL+z/utiUe3c/plcGrx+7d5XRxYlHK817Lx0+k/D7lX/6vGy017j2H+i8p833yb1x5UzQ5EcbWlo74wUiswHfPXOHPPpqjObX7CD7i+PvenLtBghPxHsh6G8MwCLTOyjAkjy8mo54ai20skJ0CLz97qN7ujp8e32svt2VmNeBtZCdWl+5KkNFIAD9KB9zLtf5YMwV+RKgQ5f6ndXc/5J0WfCe8151B9anyGsKyW+WfAtcrGAGsZE2YPQ8sw4xivh/BNXeHeGbLVAbYmKLCWwQXbtEtmsyPJMgWrUfLLQvfcXc6DWFj3v+nH6358FdUtCP7TW9qwS0u9AWmSS0uDu4QIMRux1Y+Iu/gxFg3uwaKaN7DkuNDGmw+jw1xLOdHAWmaNEbXKKY5SOKMEhJ1TT5ijlkN9AXxSg6EqJ/AZLMlF5A43KSXyDwfzTC/aZSIFX6aN5Bte8ti7y9+dufOiOhxgPryw3/D0f+XZXMpy9JIrkXmMsgjO6cbwRXznJpVpK5YFlcpBMVO2BVAhxegKhQBxxcF9O69L1Ao06QTal3sXvxnO7O6jcqq0F8+Ke8aQLFUPt6iW4WcGtREAWAa73OLlnzPf+nSc5D4Ou++tNHFDY2AYO8Le8W4+8L5HOgrSw7yAEntyqNlKsAlVyqWW1SvBzBr9aoHFoG4OrM/W92a8OWBnc3Dy8ebRFE9+ImBow3N1yrKuejdXPbvhhp8YErneIxIyLqGDlEcSRn49R4ZFvi58T4dVT2q98DmoilfBIJ5M3VXDGckR6paDJEW+RTzvLVHc0y7Ckw2I5wdBzCkORgFO+lpIgOPcgfHBpCVPzAUt6yRQZyaeUD8yutBJyAw9sPBoQTkvpkDfa9ZRo56GGXPTHz28XtcaD0V7UZMOU9EmPFoyD0V6mPa/gLrHOhnU1OmF5b/YGS7E3wqngiLg6TCHbkCiTE9ZCpTisqTFZpJaHPUTxZf/T9dT9HxmzUzCPV9nitD1qvAc5eTvruPuaHz18KyKCHy4CVoTgL3Y2oa0Aey1peK9R1tW0VmIuEX8AtmetiqyZ1TfTvOtpXZmjR5foDhgCnB4a0JD/cYhf0/BpxT4JT2EJbsDRbyC7wU0/qbSkq6/RFUEvaUltI13VuAKXLqh0RQlxgdZwrFs67v4PX76Iqx6HSiGVAtc/qKk+lPUPZJAgIqFJHrrAKaXNQcxitereV1jIxM0j2rCCdmTFNmQPZMMKypEVr9Ce+FzGMac4if6gNp44qLMqjmTjCa7GppynRGYLnYYV/u0bT9D2KUjuPBEx0M/Lhk05eR32kRXPt/Cs3/nPslnu7E34sF5N74OiCLWG/cZtDNUSzYM8sIK+LwHF036UZ1XkBGiRZ1VQ05gp+9yjPtqLEtUMqBZ6LAV1QYwC61hH2n8JaxZY024TtjNYKUtfOA+SBu0R58x9A9lKgWoSFVlKPs5Ytx03kQcZhK+UoKYEtVagtkQFlSWV36WTeAmqq9kA2aKWeh46/Ep8U5o2TIHKE73N5A7jBHzHnhv5DVz5lNpTDrGl1FP5KKs7ZG5kxL7+guGu8/jIJMkoPfLpctBptHeUqriPRMXtoZpl0uLfIPIzhavs8OSVeg4TwaJW/2V6968L+/eoUV3YFx3zgiIxnJnuylpEXqdLQivBzgg2n/a0NK6+o1k/muTjJyspim9+RUfsBDuT+3Is2KY8WODF4UpUCzwhXo9GIOS5rPsNQhKrxamP6hyquti4vRrocoW/flVYaboh1Xl5ME0Z1pg34qnj1HMQHdSsWHKeKLNidwK1EAnpIpj7YShakyZ8vrmsWLPaRVvVr65ooYd+7DLOUNSg426fL4gTyYLikigyEUXqHQpz4H/WqiiP0vTu0WVloA4+NPlZTZ0uGxFDd3WZML/K1Nm/mAqrKfWOyKrJ3S1JjU0sUykPlF4qqTOucsjGoNILGckWRKqOoY1QVoCdIpw1IptjC2rdHVmk3vhjV2TBktpseb5m7jBXC8ytpUayH0LwcbeLw49ROZjr7lv4Wvz5F/6MkAuu2vPYrfYivEi4E/zglrf4RewV+HL1Dv8qegkJcKag5uj8iXhQ87Y8gvyCmv2vNmxbWsQeCPlj9eZ76McPRHGkLEOcmSaGWwd3N36Fyf6V5fmRKwIOmv7dyFPqGPM/isL3RancUVMqL4aCeqeUKuyDUhPUtZFsD1eubnXi751c85CSAp+vlKTmZFf2LSU3LBAcMOVty23/CYTHM1y+hLd1ZGN2Rmd5zLerYXeFbAPJD2bhGDjGgxtamdDGOT5+AEPoZNV0YEu2jP445wRBl6d8fy+xiTbpUnK02QrFbonKstgt9d688XI5mn7cWta8eQmv/+g2xcs1CJxba6CXOXerQLQjyLkTKqmPjmO2kGZY4d+Tc7fgRDB54fsvN61ne1rTPfD7kpqcFcSVBRvgh+5dhmCav92LQ5LbVhlykOl21BbTzo1NRgqGmJJunRLT/SfdUVvMbcR0eThGCCpXgnqAiXd0RqWkaK1W344ngycfOAvNuKM2mcxeJ8A7olSAnFBNm3CXQyYAtcVkfkcyEyC5XL4ulY88XiIvGkgrqXdFBJStp3Ekl4/9ES9w5gSvmFJjjpKCcs+AJ22gJ2BODWR3YceSb95yzBgNB8fAGfYwnfoL3JoNLejh/YT92DyHoIPSmfQ9i71O+JJqIsWCpvmShC0G+wYaQZc2hG7crEbgaLdQAfiJ/wM= \ No newline at end of file diff --git a/doc/rtd/content/99_appendices/appendix2/sub/mlpro_bf/layer1_computation/images/MLPro-BF-MT_class_diagram.drawio b/doc/rtd/content/99_appendices/appendix2/sub/mlpro_bf/layer1_computation/images/MLPro-BF-MT_class_diagram.drawio index c3ff04c03..7981fd6d4 100644 --- a/doc/rtd/content/99_appendices/appendix2/sub/mlpro_bf/layer1_computation/images/MLPro-BF-MT_class_diagram.drawio +++ b/doc/rtd/content/99_appendices/appendix2/sub/mlpro_bf/layer1_computation/images/MLPro-BF-MT_class_diagram.drawio @@ -1,6 +1,6 @@ - + @@ -112,7 +112,7 @@ - + @@ -132,26 +132,26 @@ - - + + - + - + - + - + - + - + diff --git a/doc/rtd/content/99_appendices/appendix2/sub/mlpro_bf/layer1_computation/images/MLPro-BF-MT_class_diagram.drawio.png b/doc/rtd/content/99_appendices/appendix2/sub/mlpro_bf/layer1_computation/images/MLPro-BF-MT_class_diagram.drawio.png index 0379dba78..bf90ca455 100644 Binary files a/doc/rtd/content/99_appendices/appendix2/sub/mlpro_bf/layer1_computation/images/MLPro-BF-MT_class_diagram.drawio.png and b/doc/rtd/content/99_appendices/appendix2/sub/mlpro_bf/layer1_computation/images/MLPro-BF-MT_class_diagram.drawio.png differ diff --git a/src/mlpro/bf/mt.py b/src/mlpro/bf/mt.py index e882fa222..a03bd3099 100644 --- a/src/mlpro/bf/mt.py +++ b/src/mlpro/bf/mt.py @@ -48,10 +48,14 @@ ## -- 2025-10-09 2.6.0 DA Method Async._create_so(): additional parameter p_kwargs ## -- 2026-09-29 2.7.0 DA Method Async.__init__(): additional parameter p_kwargs ## -- 2026-10-05 2.8.0 DA Refactoring +## -- 2026-11-19 2.9.0 DA Refactoring/tuning: +## -- - Class Async: parent Log removed +## -- - Method Async._start_async(): tuning +## -- - Class Async: thread-safe task registration and waiting ## ------------------------------------------------------------------------------------------------- """ -Ver. 2.8.0 (2026-10-05) +Ver. 2.9.0 (2026-11-19) This module provides classes for multitasking with optional interprocess communication (IPC) based on shared objects. Multitasking in MLPro combines multrithreading and multiprocessing and simplifies @@ -306,7 +310,7 @@ def clear_results(self): ## ------------------------------------------------------------------------------------------------- ## ------------------------------------------------------------------------------------------------- -class Async (Range, Log): +class Async (Range): """ Property class that enables child classes to run sub-tasks asynchronously. Depending on the given range a task can be executed as a separate thread in the same process or a separate @@ -318,8 +322,6 @@ class Async (Range, Log): Maximum range of asynchonicity. See class Range. Default is Range.C_RANGE_PROCESS. p_class_shared Optional class for a shared object (class Shared or a child class of Shared) - p_logging - Log level (see constants of class Log). Default: Log.C_LOG_ALL p_kwargs : dict Optional parameters for a shared object. """ @@ -327,20 +329,21 @@ class Async (Range, Log): ## ------------------------------------------------------------------------------------------------- def __init__( self, p_range_max:int=Range.C_RANGE_PROCESS, - p_class_shared=None, - p_logging=Log.C_LOG_ALL, + p_class_shared=None, **p_kwargs ): - Log.__init__(self, p_logging=p_logging) Range.__init__(self, p_range=p_range_max) - self._async_tasks = [] - self._mpmanager = None - self._class_shared = p_class_shared + self._async_tasks = [] + self._async_tasks_lock = mt.Lock() + self._mpmanager = None + self._so : Shared = None - self._so : Shared = self._create_so( p_range = p_range_max, - p_class_shared = p_class_shared, - **p_kwargs ) + if p_class_shared is not None: + self._class_shared = p_class_shared + self._so = self._create_so( p_range = p_range_max, + p_class_shared = self._class_shared, + **p_kwargs ) ## ------------------------------------------------------------------------------------------------- @@ -453,27 +456,39 @@ class is specified, a new instance of the current class is created asynchronousl # 2 Execution depending on range of asynchronity - if range_run == self.C_RANGE_NONE: - # 2.1 Synchronous execution - p_target(**p_kwargs) + if range_run == self.C_RANGE_THREAD: - elif range_run in [ self.C_RANGE_THREAD, self.C_RANGE_PROCESS ]: - # 2.2 Asynchronous execution as separate thread or process - if range_run == self.C_RANGE_THREAD: - # 2.2.1 Preparation of a new thread - task = mt.Thread(target=p_target, kwargs=p_kwargs, group=None) + # 2.1 Execute as new thread + task = mt.Thread(target=p_target, kwargs=p_kwargs, group=None) + with self._async_tasks_lock: + self._async_tasks.append(task) + try: + task.start() + except BaseException: + self._async_tasks.remove(task) + raise - else: - # 2.2.2 Preparation of a new process - task = mp.Process(target=p_target, kwargs=p_kwargs, group=None) + elif range_run == self.C_RANGE_NONE: - # 2.2.3 Registration and start of new thread/process - self._async_tasks.append(task) - task.start() + # 2.2 Execute synchronously + p_target(**p_kwargs) + elif range_run == self.C_RANGE_PROCESS: + + # 2.3 Execute as new process + task = mp.Process(target=p_target, kwargs=p_kwargs, group=None) + with self._async_tasks_lock: + self._async_tasks.append(task) + try: + task.start() + except BaseException: + self._async_tasks.remove(task) + raise + else: - raise NotImplementedError - + # 2.4 Unknown range of asynchonicity + raise NotImplementedError + # 3 Returns actual range of asynchonicity return range_run @@ -482,11 +497,21 @@ class is specified, a new instance of the current class is created asynchronousl ## ------------------------------------------------------------------------------------------------- def wait_async_tasks(self): """ - Waits until all internal asynchonous tasks are finished. + Waits until all currently registered asynchronous tasks have finished. + + Tasks registered while waiting are included. Call this method after + submitting the initial tasks. Only one waiter per Async instance is + supported. Do not call from a task that belongs to the same instance. """ - for task in self._async_tasks: task.join() - self._async_tasks.clear() + while True: + with self._async_tasks_lock: + if not self._async_tasks: + return + + task = self._async_tasks.pop(0) + + task.join() ## ------------------------------------------------------------------------------------------------- @@ -569,10 +594,10 @@ def __init__( self, else: self.set_name(str(self.get_id())) - Async.__init__(self, p_range_max=p_range_max, p_class_shared=p_class_shared, p_logging=p_logging) - EventManager.__init__(self, p_event_config=p_event_config) - Plottable.__init__(self, p_visualize=p_visualize) - Persistent.__init__(self, p_id=p_id, p_logging=p_logging) + Async.__init__( self, p_range_max = p_range_max, p_class_shared = p_class_shared ) + EventManager.__init__( self, p_event_config = p_event_config ) + Plottable.__init__( self, p_visualize = p_visualize ) + Persistent.__init__( self, p_id = p_id, p_logging = p_logging ) self._custom_run_method = self._get_custom_run_method() self._autorun(p_autorun=p_autorun, p_kwargs=self._kwargs) diff --git a/src/mlpro/rl/pool/actionplanner/mpc.py b/src/mlpro/rl/pool/actionplanner/mpc.py index 5c1a89bc2..686f5caa6 100644 --- a/src/mlpro/rl/pool/actionplanner/mpc.py +++ b/src/mlpro/rl/pool/actionplanner/mpc.py @@ -12,10 +12,11 @@ ## -- 2023-02-04 1.1.1 SY Bug fixing ## -- 2025-04-24 1.1.2 DA Bugfix in method MPC._async_subtask() ## -- 2025-07-17 1.2.0 DA Refactoring +## -- 2026-11-19 1.3.0 DA Refactoring ## ------------------------------------------------------------------------------------------------- """ -Ver. 1.2.0 (2025-07-17) +Ver. 1.3.0 (2026-11-19) This module provides a default implementation of model predictive control (MPC). """ @@ -41,7 +42,7 @@ ## ------------------------------------------------------------------------------------------------- ## ------------------------------------------------------------------------------------------------- -class MPC(ActionPlanner, ScientificObject, mt.Async): +class MPC(ActionPlanner, ScientificObject, mt.Async, Log): """ Template class for MPC to be used as part of model-based planning agents. The goal is to find the best sequence of actions that leads to a maximum reward. @@ -78,8 +79,7 @@ def __init__(self, p_logging=p_logging) mt.Async.__init__(self, p_range_max=p_range_max, - p_class_shared=mt.Shared, - p_logging=p_logging) + p_class_shared=mt.Shared) ## ------------------------------------------------------------------------------------------------- diff --git a/test/howtos/bf/mt/howto_bf_mt_001_parallel_algorithms.py b/test/howtos/bf/mt/howto_bf_mt_001_parallel_algorithms.py index c9f6910da..c8e4e1dfd 100644 --- a/test/howtos/bf/mt/howto_bf_mt_001_parallel_algorithms.py +++ b/test/howtos/bf/mt/howto_bf_mt_001_parallel_algorithms.py @@ -9,10 +9,11 @@ ## -- 2022-10-09 1.1.0 DA Simplification ## -- 2022-10-12 1.2.0 DA Restructuring of demo steps ## -- 2022-10-13 1.3.0 DA Restructuring of demo steps +## -- 2026-11-19 1.4.0 DA Refactoring ## ------------------------------------------------------------------------------------------------- """ -Ver. 1.3.0 (2022-10-13) +Ver. 1.4.0 (2026-11-19) This module demonstrates the use of classes ASync and Shared as part of MLPro's multitasking concept. Both classes are used to implement a simple parallel algorithm class MyParallelAlgorithm with a @@ -51,7 +52,7 @@ ## ------------------------------------------------------------------------------------------------- ## ------------------------------------------------------------------------------------------------- -class MyParallelAlgorithm (mt.Async): +class MyParallelAlgorithm (mt.Async, Log): """ This class demonstrates how to run methods asynchronously and to collect results in a shared object. @@ -69,9 +70,10 @@ def __init__( self, p_class_shared=mt.Shared, p_logging=Log.C_LOG_ALL ): - super().__init__( p_range_max=p_range_max, - p_class_shared=p_class_shared, - p_logging=p_logging ) + mt.Async.__init__( self, + p_range_max=p_range_max, + p_class_shared=p_class_shared ) + Log.__init__( self, p_logging = p_logging ) self._num_tasks = p_num_tasks self._duration = p_duration