Future pending attached to a different loop

[mattgallivan]

Thanks for this library!

I'm running into some troubles sending data to the hello script example.

The following is my Blueprint setup:

I start the server and receive the following log:

connect nu7lldCxrFrDRtNcAAAR
loading
loaded.
hello on_setup
started.
sendInput: {'inputData': {'number': 0.3581652343273163}, 'targetFunction': 'on_json_input'}
{'number': 0.3581652343273163}
run_on_gt callback: {}
sendInput return: {}
{}
Task exception was never retrieved
future: <Task finished name='Task-171' coro=<AsyncServer._handle_event_internal() done, defined at /mnt/c/Test/Plugins/Marketplace/MachineLearningRemote-Unreal/Server/ml-remote-server/env/lib/python3.8/site-packages/socketio/asyncio_server.py:521> exception=RuntimeError("Task <Task pending name='Task-171' coro=<AsyncServer._handle_event_internal() running at /mnt/c/Test/Plugins/Marketplace/MachineLearningRemote-Unreal/Server/ml-remote-server/env/lib/python3.8/site-packages/socketio/asyncio_server.py:523>> got Future attached to a different loop")>
Traceback (most recent call last):
File "/mnt/c/Test/Plugins/Marketplace/MachineLearningRemote-Unreal/Server/ml-remote-server/env/lib/python3.8/site-packages/socketio/asyncio_server.py", line 523, in _handle_event_internal
r = await server._trigger_event(data[0], namespace, sid, *data[1:])
File "/mnt/c/Test/Plugins/Marketplace/MachineLearningRemote-Unreal/Server/ml-remote-server/env/lib/python3.8/site-packages/socketio/asyncio_server.py", line 557, in _trigger_event
ret = await handler(*args)
File "server.py", line 88, in send_input
return await future
RuntimeError: Task <Task pending name='Task-171' coro=<AsyncServer._handle_event_internal() running at /mnt/c/Test/Plugins/Marketplace/MachineLearningRemote-Unreal/Server/ml-remote-server/env/lib/python3.8/site-packages/socketio/asyncio_server.py:523>> got Future attached to a different loop

It seems to receive the input once and then throws an issue concerning "got Future attached to a different loop". Any ideas why this might be the case?

[reidsanders]

Same issue here.

[reidsanders]

Some debugging attempts thwarted. If this is working for anyone please post your environment and commits for both plugins so I can try them. Thank You!

I'm testing this with just the "empty_example" script as in the Readme.

Originally it had been working at
SocketIOClient-Unreal tags/v1.10.0 e23640f
MachineLearningRemote-Unreal at c2fc560
UE 4.27.2

After a while (and a windows reinstall) I got the future error as above. I assumed that meant I could find some working combination of versions or libraries but that doesn't seem to be the case.

I'm running windows 10 latest updates.
I tried both UE 4.27.2 and UE 5.0.3
I am running the ml-remote-server/server.py at the included pinned commit . I also tried the current master.

I tried various combinations of MachineLearningRemote-Unreal and SocketIOClient-Unreal and could compile at:
SocketIO checkout at 1719eecf
MachineLearningRemote at bb1a988

with same future error eg

Task exception was never retrieved
future: <Task finished name='Task-23' coro=<AsyncServer._handle_event_internal() done, defined at C:\Users\reids\miniconda3\envs\mlremote2\lib\site-packages\socketio\asyncio_server.py:522> exception=RuntimeError("Task <Task pending name='Task-23' coro=<AsyncServer._handle_event_internal() running at C:\\Users\\reids\\miniconda3\\envs\\mlremote2\\lib\\site-packages\\socketio\\asyncio_server.py:524>> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "C:\Users\reids\miniconda3\envs\mlremote2\lib\site-packages\socketio\asyncio_server.py", line 524, in _handle_event_internal
    r = await server._trigger_event(data[0], namespace, sid, *data[1:])
  File "C:\Users\reids\miniconda3\envs\mlremote2\lib\site-packages\socketio\asyncio_server.py", line 558, in _trigger_event
    ret = await handler(*args)
  File "f:\reids\Documents\UnrealProjects\MLExperiments\Plugins\MachineLearningRemote-Unreal\Server\ml-remote-server\server.py", line 88, in send_input
    return await future
RuntimeError: Task <Task pending name='Task-23' coro=<AsyncServer._handle_event_internal() running at C:\Users\reids\miniconda3\envs\mlremote2\lib\site-packages\socketio\asyncio_server.py:524>> got Future <Future pending> attached to a different loop

With current master in both plugins:
SocketIOClientUnreal: fd7cb7aec1
MachineLearningRemote-Unreal: 18b5e7d

Compilation failed with

1>[1/4] Compile Module.MachineLearningRemote.cpp
1>F:\reids\Documents\UnrealProjects\Seance\Plugins\machine-learning-remote-ue4\Source\MachineLearningRemote\Private\MachineLearningRemoteComponent.cpp(65): error C2679: binary '=': no operator found which takes a right-hand operand of type 'UMachineLearningRemoteComponent::BeginPlay::<lambda_325db2c4b5c34669ac6a24a35a692c7f>' (or there is no acceptable conversion)
1>C:\Program Files\Epic Games\UE_5.0\Engine\Source\Runtime\Core\Public\Templates\Function.h(971): note: could be 'TFunction<void (const FString &,const FString &)> &TFunction<void (const FString &,const FString &)>::operator =(const TFunction<void (const FString &,const FString &)> &)'
1>C:\Program Files\Epic Games\UE_5.0\Engine\Source\Runtime\Core\Public\Templates\Function.h(962): note: or       'TFunction<void (const FString &,const FString &)> &TFunction<void (const FString &,const FString &)>::operator =(TFunction<void (const FString &,const FString &)> &&)'
1>F:\reids\Documents\UnrealProjects\Seance\Plugins\machine-learning-remote-ue4\Source\MachineLearningRemote\Private\MachineLearningRemoteComponent.cpp(77): note: while trying to match the argument list '(TFunction<void (const FString &,const FString &)>, UMachineLearningRemoteComponent::BeginPlay::<lambda_325db2c4b5c34669ac6a24a35a692c7f>)'
1>F:\reids\Documents\UnrealProjects\Seance\Plugins\machine-learning-remote-ue4\Source\MachineLearningRemote\Private\MachineLearningRemoteComponent.cpp(136): error C2039: 'ClearCallbacks': is not a member of 'FSocketIONative'

ClearCallbacks seems to have been changed to ClearAllCallbacks which is an easy fix. However I do not know how to address ue4\Source\MachineLearningRemote\Private\MachineLearningRemoteComponent.cpp(65): error C2679: binary '=': no operator found which takes a right-hand operand of type 'UMachineLearningRemoteComponent::BeginPlay::<lambda_325db2c4b5c34669ac6a24a35a692c7f>'

Trying more combinations:
SocketIOClientUnreal: a5fac1b274
MachineLearningRemote-Unreal: 18b5e7d (master)

1>[21/24] Link UnrealEditor-MachineLearningRemote.dll

1>Module.MachineLearningRemote.cpp.obj : error LNK2001: unresolved external symbol "struct FLogCategoryMLBaseLog MLBaseLog" (?MLBaseLog@@3UFLogCategoryMLBaseLog@@A)
1>F:\reids\Documents\UnrealProjects\Seance\Plugins\machine-learning-remote-ue4\Binaries\Win64\UnrealEditor-MachineLearningRemote.dll : fatal error LNK1120: 1 unresolved externals

This seems to have been introduced in the fix for #13
With git bisect I get various errors, but the most up to date commits that compile seem to be
SocketIOClientUnreal: f3828dfa3
MachineLearningRemote-Unreal: bb1a988

which still produce the same future error.

---

conda environment.yml in case that's helpful:

name: mlremote2
channels:
  - pytorch
  - conda-forge
  - defaults
dependencies:
  - blas=2.116=mkl
  - blas-devel=3.9.0=16_win64_mkl
  - brotlipy=0.7.0=py310he2412df_1004
  - bzip2=1.0.8=he774522_0
  - ca-certificates=2022.9.24=h5b45459_0
  - certifi=2022.9.24=pyhd8ed1ab_0
  - cffi=1.15.1=py310hcbf9ad4_0
  - charset-normalizer=2.1.1=pyhd8ed1ab_0
  - cryptography=37.0.4=py310ha857299_0
  - cudatoolkit=11.6.0=hc0ea762_10
  - freetype=2.10.4=h546665d_1
  - idna=3.4=pyhd8ed1ab_0
  - intel-openmp=2022.1.0=h57928b3_3787
  - jpeg=9e=h8ffe710_2
  - libblas=3.9.0=16_win64_mkl
  - libcblas=3.9.0=16_win64_mkl
  - libffi=3.4.2=hd77b12b_4
  - liblapack=3.9.0=16_win64_mkl
  - liblapacke=3.9.0=16_win64_mkl
  - libpng=1.6.37=h1d00b33_2
  - libtiff=4.2.0=h0c97f57_3
  - libuv=1.44.2=h8ffe710_0
  - libwebp=1.2.4=h8ffe710_0
  - libwebp-base=1.2.4=h8ffe710_0
  - lz4-c=1.9.3=h8ffe710_1
  - m2w64-gcc-libgfortran=5.3.0=6
  - m2w64-gcc-libs=5.3.0=7
  - m2w64-gcc-libs-core=5.3.0=7
  - m2w64-gmp=6.1.0=2
  - m2w64-libwinpthread-git=5.0.0.4634.697f757=2
  - mkl=2022.1.0=h6a75c08_874
  - mkl-devel=2022.1.0=h57928b3_875
  - mkl-include=2022.1.0=h6a75c08_874
  - msys2-conda-epoch=20160418=1
  - numpy=1.23.2=py310h8a5b91a_0
  - openssl=1.1.1q=h8ffe710_0
  - pillow=9.2.0=py310hdc2b20a_1
  - pip=22.2.2=py310haa95532_0
  - pycparser=2.21=pyhd8ed1ab_0
  - pyopenssl=22.0.0=pyhd8ed1ab_1
  - pysocks=1.7.1=pyh0701188_6
  - python=3.10.6=hbb2ffb3_0
  - python_abi=3.10=2_cp310
  - pytorch=1.12.1=py3.10_cuda11.6_cudnn8_0
  - pytorch-mutex=1.0=cuda
  - requests=2.28.1=pyhd8ed1ab_1
  - setuptools=63.4.1=py310haa95532_0
  - sqlite=3.39.3=h2bbff1b_0
  - tbb=2021.5.0=h2d74725_1
  - tk=8.6.12=h2bbff1b_0
  - torchaudio=0.12.1=py310_cu116
  - torchvision=0.13.1=py310_cu116
  - typing_extensions=4.4.0=pyha770c72_0
  - tzdata=2022e=h04d1e81_0
  - urllib3=1.26.11=pyhd8ed1ab_0
  - vc=14.2=h21ff451_1
  - vs2015_runtime=14.27.29016=h5e58377_2
  - wheel=0.37.1=pyhd3eb1b0_0
  - win_inet_pton=1.1.0=py310h5588dad_4
  - wincertstore=0.2=py310haa95532_2
  - xz=5.2.6=h8cc25b3_0
  - zlib=1.2.12=h8cc25b3_3
  - zstd=1.5.0=h6255e5f_0
  - pip:
    - absl-py==1.3.0
    - aiohttp==3.8.3
    - aiosignal==1.2.0
    - annoy==1.17.1
    - asttokens==2.0.8
    - astunparse==1.6.3
    - async-timeout==4.0.2
    - attrs==22.1.0
    - backcall==0.2.0
    - bidict==0.22.0
    - blis==0.7.9
    - cachetools==5.2.0
    - catalogue==2.0.8
    - chardet==3.0.4
    - click==8.1.3
    - colorama==0.4.5
    - confection==0.0.3
    - cymem==2.0.7
    - decorator==5.1.1
    - executing==1.1.1
    - filelock==3.8.0
    - flatbuffers==22.9.24
    - frozenlist==1.3.1
    - gast==0.4.0
    - google-auth==2.13.0
    - google-auth-oauthlib==0.4.6
    - google-pasta==0.2.0
    - grpcio==1.50.0
    - h5py==3.7.0
    - halo==0.0.31
    - hnswlib==0.6.2
    - huggingface-hub==0.10.1
    - ipython==8.5.0
    - jedi==0.18.1
    - jinja2==3.1.2
    - joblib==1.2.0
    - keras==2.10.0
    - keras-preprocessing==1.1.2
    - langcodes==3.3.0
    - libclang==14.0.6
    - log-symbols==0.0.14
    - markdown==3.4.1
    - markupsafe==2.1.1
    - matplotlib-inline==0.1.6
    - multidict==4.7.6
    - murmurhash==1.0.9
    - nltk==3.7
    - oauthlib==3.2.2
    - opt-einsum==3.3.0
    - packaging==21.3
    - pandas==1.5.1
    - parse==1.19.0
    - parso==0.8.3
    - pathy==0.6.2
    - pickleshare==0.7.5
    - preshed==3.0.8
    - prompt-toolkit==3.0.31
    - protobuf==3.19.6
    - pure-eval==0.2.2
    - pyasn1==0.4.8
    - pyasn1-modules==0.2.8
    - pyaudio==0.2.12
    - pybase64==1.2.3
    - pydantic==1.10.2
    - pygments==2.13.0
    - pyparsing==3.0.9
    - pysoundfile==0.9.0.post1
    - python-dateutil==2.8.2
    - python-engineio==4.3.4
    - python-socketio==5.7.2
    - pytz==2022.5
    - pyyaml==6.0
    - regex==2022.9.13
    - requests-oauthlib==1.3.1
    - rsa==4.9
    - scikit-learn==1.1.2
    - scipy==1.9.3
    - sentence-transformers==2.2.2
    - sentencepiece==0.1.97
    - six==1.16.0
    - smart-open==5.2.1
    - spacy==3.4.2
    - spacy-legacy==3.0.10
    - spacy-loggers==1.0.3
    - spinners==0.0.24
    - srsly==2.4.5
    - stack-data==0.5.1
    - tensorboard==2.10.1
    - tensorboard-data-server==0.6.1
    - tensorboard-plugin-wit==1.8.1
    - tensorflow==2.10.0
    - tensorflow-estimator==2.10.0
    - tensorflow-io-gcs-filesystem==0.27.0
    - termcolor==2.0.1
    - thinc==8.1.5
    - threadpoolctl==3.1.0
    - tokenizers==0.13.1
    - tqdm==4.64.1
    - traitlets==5.5.0
    - transformers==4.23.1
    - typer==0.4.2
    - wasabi==0.10.1
    - wave==0.0.2
    - wcwidth==0.2.5
    - webrtcvad==2.0.10
    - werkzeug==2.2.2
    - wrapt==1.14.1
    - yarl==1.8.1
prefix: C:\Users\reids\miniconda3\envs\mlremote2

[reidsanders]

If it is working for anyone I would greatly appreciate if you can post your commits / library versions. Thanks!