web-dev-qa-db-fra.com

Le multitraitement provoque le blocage de Python) et indique qu'une erreur était peut-être en cours dans un autre thread lors de l'appel de fork ()

Je suis relativement nouveau sur Python) et j'essaie d'implémenter un module de multitraitement pour ma boucle for.

J'ai un tableau d'URL d'images stocké dans img_urls que je dois télécharger et appliquer à Google Vision.

if __name__ == '__main__':

    img_urls = [ALL_MY_Image_URLS]
    runAll(img_urls)
    print("--- %s seconds ---" % (time.time() - start_time))

Ceci est ma méthode runAll ()

def runAll(img_urls):
    num_cores = multiprocessing.cpu_count()

    print("Image URLS  {}",len(img_urls))
    if len(img_urls) > 2:
        numberOfImages = 0
    else:
        numberOfImages = 1

    start_timeProcess = time.time()

    pool = multiprocessing.Pool()
    pool.map(annotate,img_urls)
    end_timeProcess = time.time()
    print('\n Time to complete ', end_timeProcess-start_timeProcess)

    print(full_matching_pages)


def annotate(img_path):
    file =  requests.get(img_path).content
    print("file is",file)
    """Returns web annotations given the path to an image."""
    print('Process Working under ',os.getpid())
    image = types.Image(content=file)
    web_detection = vision_client.web_detection(image=image).web_detection
    report(web_detection)

Je reçois ceci comme avertissement quand je le lance et python plante)

objc[67570]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called.
objc[67570]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
objc[67567]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called.
objc[67567]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
objc[67568]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called.
objc[67568]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
objc[67569]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called.
objc[67569]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
objc[67571]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called.
objc[67571]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
objc[67572]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called.
objc[67572]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
pythonmultithreadingpython-3.x
34
SriTeja Chilakamarri

Cette erreur est due à une sécurité accrue pour limiter le multithreading dans Mac OS High Sierra. Je sais que cette réponse est un peu en retard, mais j'ai résolu le problème en utilisant la méthode suivante:

Définissez une variable d'environnement .bash_profile pour autoriser les applications ou les scripts multithreads dans le cadre des nouvelles règles de sécurité Mac OS High Sierra.

Ouvrez un terminal:

$ nano .bash_profile

Ajoutez la ligne suivante à la fin du fichier:

export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

Enregistrez, quittez, fermez le terminal et rouvrez le terminal. Vérifiez que la variable d'environnement est maintenant définie:

$ env

Vous verrez une sortie semblable à:

TERM_PROGRAM=Apple_Terminal
Shell=/bin/bash
TERM=xterm-256color
TMPDIR=/var/folders/pn/vasdlj3ojO#OOas4dasdffJq/T/
Apple_PubSub_Socket_Render=/private/tmp/com.Apple.launchd.E7qLFJDSo/Render
TERM_PROGRAM_VERSION=404
TERM_SESSION_ID=NONE
OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

Vous devriez maintenant pouvoir exécuter votre script python avec multithreading.

79
jonnyjandles