Skip to content

Magento 2.4.7-p6 Reindexing takes hours when running at night

Every night over server goes offline due to load.
I reindex every night at midnight from a cron job that runs

bin/magento indexer:reindex

Then at 2am I run

bin/magento cron:run

I have attached a image indexer settings
enter image description here

Here is the cron scheduled tasks
Cron Scheduled Tasks

group:index
Generate Schedules Every: 30
Schedule Ahead: 1
Missed if Not run withn: 45
History Cleanup every: 1440
Success history lifetime: 1440
failure history lifetime: 1440
Use separate process: yes

group: default
Generate Schedules Every: 30
Schedule Ahead: 1
Missed if Not run withn: 45
History Cleanup every: 10
Success history lifetime: 60
failure history lifetime: 600
Use separate process: no

group: consumers
Generate Schedules Every: 30
Schedule Ahead: 1
Missed if Not run withn: 45
History Cleanup every: 10
Success history lifetime: 60
failure history lifetime: 600
Use separate process: yes

group:saas_data_exporter
Generate Schedules Every: 30
Schedule Ahead: 1
Missed if Not run withn: 45
History Cleanup every: 1440
Success history lifetime: 1440
failure history lifetime: 1440
Use separate process: yes

group: resync_failed_feeds_data_exporter
Generate Schedules Every: 30
Schedule Ahead: 1
Missed if Not run withn: 45
History Cleanup every: 1440
Success history lifetime: 1440
failure history lifetime: 1440
Use separate process: yes

group:payment_services_order_sync
Generate Schedules Every: 5
Schedule Ahead: 1
Missed if Not run withn: 30
History Cleanup every: 1440
Success history lifetime: 1440
failure history lifetime: 1440
Use separate process: yes

group:payment_services_data_export
Generate Schedules Every: 30
Schedule Ahead: 1
Missed if Not run withn: 45
History Cleanup every: 1440
Success history lifetime: 1440
failure history lifetime: 1440
Use separate process: yes

Once the server backup kicks in along with the reindexing and cron it knocks the server offline.
There are a lot of categories and products on the website.

Is there anything I can do to improve reindexing time and do ireally need to run a cron job for cron at 2am.

Any Help would be great.

Thanks
Neit