stoffelmauro/ConsultancyProject_2_ETL
1
0
Fork 1
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Fehlermeldungen bei moving average #13

New Issue
Closed
opened 2025-01-13 21:33:47 +01:00 by dianigionath · 1 comment
dianigionath commented 2025-01-13 21:33:47 +01:00
Collaborator
Copy Link

Wenn ich alle scrapedate durchgehe, erhalte ich ziemlich viele null Werte. Ist das so vorgesehen?
grafik

Bei denen welche null lieferen schmeisst es jeweils eine Fehlermeldung z. B.:

polars.exceptions.ShapeError: unable to add a column of length 6 to a DataFrame of height 0
INFO:     127.0.0.1:37078 - "GET /region/3/movingAverage/2024-08-01 HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/fastapi/routing.py", line 301, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/fastapi/routing.py", line 214, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2441, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 943, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/src/api/main.py", line 98, in region_capacities_data
    result = etl_rmA.region_movingAverage(id, startDate)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/src/data/etl_region_movAverage.py", line 111, in region_movingAverage
    outDF = outDF.with_columns(moving_averages=pl.Series(moving_averages))
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/polars/dataframe/frame.py", line 9194, in with_columns
    return self.lazy().with_columns(*exprs, **named_exprs).collect(_eager=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/polars/lazyframe/frame.py", line 2055, in collect
    return wrap_df(ldf.collect(callback))
                   ^^^^^^^^^^^^^^^^^^^^^
Wenn ich alle scrapedate durchgehe, erhalte ich ziemlich viele null Werte. Ist das so vorgesehen? ![grafik](/attachments/0bc01057-7151-44ce-8c12-fe9904c2290d) Bei denen welche null lieferen schmeisst es jeweils eine Fehlermeldung z. B.: ```bash polars.exceptions.ShapeError: unable to add a column of length 6 to a DataFrame of height 0 INFO: 127.0.0.1:37078 - "GET /region/3/movingAverage/2024-08-01 HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__ await self.middleware_stack(scope, receive, send) File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__ raise exc File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__ await self.app(scope, receive, _send) File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app raise exc File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/routing.py", line 715, in __call__ await self.middleware_stack(scope, receive, send) File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/routing.py", line 735, in app await route.handle(scope, receive, send) File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle await self.app(scope, receive, send) File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/routing.py", line 76, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app raise exc File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/routing.py", line 73, in app response = await f(request) ^^^^^^^^^^^^^^^^ File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/fastapi/routing.py", line 301, in app raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/fastapi/routing.py", line 214, in run_endpoint_function return await run_in_threadpool(dependant.call, **values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool return await anyio.to_thread.run_sync(func, *args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync return await get_async_backend().run_sync_in_worker_thread( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2441, in run_sync_in_worker_thread return await future ^^^^^^^^^^^^ File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 943, in run result = context.run(func, *args) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/gio/Code/ConsultancyProject_2_ETL/etl/src/api/main.py", line 98, in region_capacities_data result = etl_rmA.region_movingAverage(id, startDate) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/gio/Code/ConsultancyProject_2_ETL/etl/src/data/etl_region_movAverage.py", line 111, in region_movingAverage outDF = outDF.with_columns(moving_averages=pl.Series(moving_averages)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/polars/dataframe/frame.py", line 9194, in with_columns return self.lazy().with_columns(*exprs, **named_exprs).collect(_eager=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/gio/Code/ConsultancyProject_2_ETL/etl/.pixi/envs/default/lib/python3.12/site-packages/polars/lazyframe/frame.py", line 2055, in collect return wrap_df(ldf.collect(callback)) ^^^^^^^^^^^^^^^^^^^^^ ```
grafik.png
223 KiB
stoffelmauro was assigned by dianigionath 2025-01-13 21:33:47 +01:00
stoffelmauro commented 2025-01-13 23:07:15 +01:00
Owner
Copy Link

Neue Version mit verbesserter Logik gepusht, bitte überprüfen.

Neue Version mit verbesserter Logik gepusht, bitte überprüfen.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
bollicolin dianigionath winklecharle stoffelmauro
No Assignees stoffelmauro
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: stoffelmauro/ConsultancyProject_2_ETL#13
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?