Multi-Core Python: multiprocessing Vs. zeroMQ? -


i'd write python code loads data set input , analyzes it.

there 5 parallel processes analyze data -- each process processing data differently -- in unique way.

before processing begin, master script (the 1 'start' different sub-processes) define empty list.

i'd different processes write output same list mentioned above (meaning, each 1 of processes able directly manipulate list defined in master script).

meaning, if process1 changed first value of list, other processes (while running) see first value of list has changed.

i sense 2 different python modules can used solve problem: multiprocessing , zeromq.

are there reasons prefer 1 vs. other in case? answer change if -- instead of running master script on same server -- splitting processes between different (multiple) servers?

(if @ matters, using linux)

if these 2 choices , know sure you're going distributing load across multiple machines, zeromq 1 of 2 choices fits bill.

the python multiprocessing module distributing load across processes/cores on single machine. far know, there no networked protocol underlying multiprocessing module , indicated first paragraph in accompanying documentation.

zeromq can used similar inter-process messaging on single machine ipc protocol, has network-based protocols allows send messages between processes running on different machines well.

that said, question has slight tinge of xy problem since seem have arbitrarily narrowed choices 2 of many, many possibilities implementing distributed program using python.

edit answer here incorrect , can't delete accepted answer, converting wiki in case wants correct it. short story misread documentation in haste. python multiprocessing support inter-process communication on network boundary. 1 major difference zeromq zeromq designed platform agnostic mix client/server agents on different platforms whereas python multiprocessing batteries included option if client/server processes coupled python.


Comments

Post a Comment

Popular posts from this blog

c++ - Is it possible to compile a VST on linux? -

for class project i'm attempting write vst plugin backed cuda. current cuda workflow on linux box, i'd prefer compile , link there. according wikipedia , should possible (i couldn't find steinberg documentation relevant linux) can't find makefile or instructions on how build if aren't using xcode or visual studio. i'm vst 3 sdk doesn't support linux. when try compile plugin under linux, error: ./base/source/fatomic.cpp:39:30: fatal error: libkern/osatomic.h: no such file or directory this issue caused following code in "vst3 sdk/base/source/fatomic.cpp" #if mac #include <libkern/osatomic.h> #if mac_os_x_version_min_required > mac_os_x_version_10_4 #define native_atomic_type (volatile int32_t*) #else #define native_atomic_type (int32_t*) #endif #elif windows #include <windows.h> #endif but hope compiling under linux work vst sdk 2.4. reading. jvstwrapper seems run on lin...
Read more

java - Output of Eclipse is rubbish -

i want write program in eclipse language different english. when display output using system.out program, displays output correctly in console editor. when accept input console , output it, output rubbish. how can solve it. many thnaks i'm not sure if problem, have character encoding problem. make sure you're using unicode, utf-8. you can read more @ official docs converting non-unicode text .
Read more

jquery - Confused with JSON data and normal data in Django ajax request -

i read json internet still have not got grasp of it. reading article http://webcloud.se/log/ajax-in-django-with-jquery/ i not understood first part function using json def xhr_test(request, format): if request.is_ajax(): if format == 'xml': mimetype = 'application/xml' if format == 'json': mimetype = 'application/javascript' data = serializers.serialize(format, examplemodel.objects.all()) return httpresponse(data,mimetype) # if want prevent non xhr calls else: return httpresponse(status=400) my main problems are from function getting format variable does format json mean data given function json or data recived json can give me simple example ouput of function data = serializers.serialize(format, examplemodel.objects.all()) how use data when response in jquery function if don't use json in above function how input , response chnage thanks ...
Read more