[python pandas/dataframe/fastapi] ValueError: Out of range float values are not JSON compliant



1. Problem

When i was using python & fastapi & pandas, i saw the problem.

Traceback (most recent call last):
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 404, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/fastapi/applications.py", line 270, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 75, in __call__
    raise exc
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 64, in __call__
    await self.app(scope, receive, sender)
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/starlette/routing.py", line 680, in __call__
    await route.handle(scope, receive, send)
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/starlette/routing.py", line 275, in handle
    await self.app(scope, receive, send)
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/starlette/routing.py", line 65, in app
    response = await func(request)
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/fastapi/routing.py", line 260, in app
    response = actual_response_class(content, **response_args)
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/starlette/responses.py", line 192, in __init__
    super().__init__(content, status_code, headers, media_type, background)
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/starlette/responses.py", line 54, in __init__
    self.body = self.render(content)
  File "/Users/python/fastapi_server/venv/lib/python3.10/site-packages/starlette/responses.py", line 195, in render
    return json.dumps(
  File "/usr/local/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/local/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
ValueError: Out of range float values are not JSON compliant

2. Solution

When i see this error, i found the cause of problem. If i use pandas and field value is NaN, that makes problem. Datafield’s NaN serialize_json function (in /core/json_encoder.py) makes error.

So, we have to change Nan to another value. In my case, i change the value from Nan to ‘‘(empty string).

df = df.fillna('')

I hope you solve the problme using this way.