To identify the cause of the problem, refer to the log file for the detailed error message and full stack trace. If seeking community help, providing these details initially can save a lot of time.
Locate the log file
Go to the directory where you click Crynux Node.exe, there is a sub directory with name data, and inside data folder there is a folder with name logs, all the log files can be found inside.
Open a Finder window, go to the Applications folder, right click on the Crynux Node.app and select Show Package Content, then go to the folder Contents/Resources/data/logs, and the log files are located inside.
Find the logs in the container output
Find the container name of the Crynux Node:
$ docker ps
The output should be similar to:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
77e559a0d707 ghcr.io/crynux-ai/crynux-node:2.0.4 "bash start.sh run" 33 minutes ago Up 32 minutes 127.0.0.1:7412->7412/tcp ecstatic_chatterjee
In this case, the container name is ecstatic_chatterjee.
In a terminal, type in the following command:
$ docker logs {container_name}
If you want to save the logs to a file, use the following command:
$ docker logs {container_name} >> crynux.log
Find the log file inside the container
The log file can also be found under /app/logs inside the container.
If you downloaded the binary release version of Linux server, the log files can be found in the logs folder of the project root.
The log file is located at logs/crynux-server.log, relative to the project root folder.
There are several log files inside the log folder. The content of each file is described below:
crynux-server.log: Node manager related logs.
crynux-worker.log: Task executor related logs.
crynux_worker_inference.log: Task execution logs.
crynux_worker_prefetch.log: Model downloading logs.
main.log: GUI related logs. Not available on Docker versions.
Most of the error messages could be identified in the first two log files: crynux-server.log and crynux-worker.log.
Locate the error message
Open the log file in a text editor. Navigate to the time where you encountered the error, and find the lines with [ERROR], which is usually the error message. And there will be a stack trace around the error message. If you are asking for help, remember to provide the full stack trace from the first line to the last.
Here is an example of a log file with error message and the stack trace:
[2024-05-15 18:08:27] [INFO ] crynux_worker.prefetch: Start worker process: worker, data/huggingface, data/external
[2024-05-15 18:08:27] [INFO ] crynux_worker.prefetch: Start prefetching models
[2024-05-15 18:08:35] [ERROR ] crynux_server.node_manager.node_manager: Node manager init error: init task cancelled
Traceback (most recent call last):
File "D:\Crynux Node\_internal\crynux_server\node_manager\node_manager.py", line 454, in _run
async with create_task_group() as init_tg:
File "anyio\_backends\_asyncio.py", line 597, in __aexit__
File "anyio\_backends\_asyncio.py", line 668, in task_done
File "D:\Crynux Node\_internal\crynux_server\node_manager\node_manager.py", line 262, in _init
async for attemp in AsyncRetrying(
File "tenacity\_asyncio.py", line 71, in __anext__
File "tenacity\__init__.py", line 314, in iter
File "concurrent\futures\_base.py", line 449, in result
File "concurrent\futures\_base.py", line 401, in __get_result
File "D:\Crynux Node\_internal\crynux_server\node_manager\node_manager.py", line 269, in _init
await to_thread.run_sync(
File "anyio\to_thread.py", line 33, in run_sync
File "anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
asyncio.exceptions.CancelledError
[2024-05-15 18:08:35] [INFO ] crynux_server.node_manager.state_manager: Node status is NodeStatus.Stopped, cannot leave the network automatically
Traceback (most recent call last):
File "D:\Crynux Node\_internal\crynux_server\node_manager\node_manager.py", line 454, in _run
async with create_task_group() as init_tg:
File "anyio\_backends\_asyncio.py", line 597, in __aexit__
File "anyio\_backends\_asyncio.py", line 668, in task_done
File "D:\Crynux Node\_internal\crynux_server\node_manager\node_manager.py", line 262, in _init
async for attemp in AsyncRetrying(
File "tenacity\_asyncio.py", line 71, in __anext__
File "tenacity\__init__.py", line 314, in iter
File "concurrent\futures\_base.py", line 449, in result
File "concurrent\futures\_base.py", line 401, in __get_result
File "D:\Crynux Node\_internal\crynux_server\node_manager\node_manager.py", line 269, in _init
await to_thread.run_sync(
File "anyio\to_thread.py", line 33, in run_sync
File "anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
asyncio.exceptions.CancelledError
