Python Playwright memory overlad

Question

I made a code that scrapy a website continuously and after several times a got this message

<--- Last few GCs --->

[17744:00000270608DE2C0] 16122001 ms: Scavenge 2023.5 (2082.0) ->
2017.3 (2082.5) MB, 3.6 / 0.1 ms  (average mu = 0.908, current mu = 0.941) task [17744:00000270608DE2C0] 16122645 ms: Scavenge 2023.9 (2082.5) -> 2017.7 (2083.0) MB, 3.5 / 0.0 ms  (average mu = 0.908, current mu = 0.941) task  [17744:00000270608DE2C0] 16128334 ms: Scavenge 2024.1 (2083.0) -> 2017.7 (2099.0) MB, 4.7 / 0.0 ms  (average mu = 0.908, current mu = 0.941) task 


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory  1: 00007FF66A07013F v8::internal::CodeObjectRegistry::~CodeObjectRegistry+112495  2: 00007FF669FFF396 DSA_meth_get_flags+65526  3: 00007FF66A00024D node::OnFatalError+301  4: 00007FF66A9319EE v8::Isolate::ReportExternalAllocationLimitReached+94  5: 00007FF66A91BECD v8::SharedArrayBuffer::Externalize+781  6: 00007FF66A7BF61C v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1468  7: 00007FF66A7BC754 v8::internal::Heap::CollectGarbage+4244  8: 00007FF66A76C3B5 v8::internal::IndexGenerator::~IndexGenerator+22165  9: 00007FF669F90E9F v8::CFunctionInfo::HasOptions+22111 10: 00007FF669F8F6B6 v8::CFunctionInfo::HasOptions+15990 11: 00007FF66A0CF25B uv_async_send+331 12: 00007FF66A0CE9EC uv_loop_init+1292 13: 00007FF66A0CEB8A uv_run+202 14: 00007FF66A09DC95 node::SpinEventLoop+309 15: 00007FF669FB7AC3 cppgc::internal::NormalPageSpace::linear_allocation_buffer+53827 16: 00007FF66A034FBD node::Start+221 17: 00007FF669E588CC RC4_options+348108 18: 00007FF66AEB08F8 v8::internal::compiler::RepresentationChanger::Uint32OverflowOperatorFor+14472 19: 00007FFEB62C7034 BaseThreadInitThunk+20 20: 00007FFEB78A2651 RtlUserThreadStart+33

And after that my code stop work. Does anyone who has had this problem know how to solve it? I'm using python 3.8.8 and playwright 1.22.0

And I'm imported this libray to make the webpage

    from playwright.sync_api import sync_playwright

Thanks guys!

Answer 1

Please refer this stack overflow link for more details. hope this will help to solve your issue