Install

create database example-update-massive

configure .env

DB_CONNECTION=pgsql
DB_HOST=postgres
DB_PORT=5432
DB_DATABASE=example-update-massive
DB_USERNAME=default
DB_PASSWORD=secret

QUEUE_CONNECTION=database

ITEMS_COUNT=10000

ITEMS_COUNT é a variavel que irá definir quantos registros a fatory irá criar no banco de dados.

run migrate;

php artisan migrate:fresh --seed

Insert

Inserir um registro por vez.

Insert1

insert into transactions (id, value, date)
values (1, 10, '2023-04-01');

insert into transactions (id, value, date)
values (2, 20, '2023-04-02');

insert into transactions (id, value, date)
values (3, 30, '2023-04-03');

Insert2

insert into transactions (id, value, date)
values (1, 10, '2023-04-01'),
       (2, 20, '2023-04-02'),
       (3, 30, '2023-04-03'), ....;

Nesse modelo, existe uma limitação:

SQLSTATE[HY000]: General error: 7 number of parameters must be between 0 and 65535

Insert3

insert into transactions (id, value, date)
values (1, 10, '2023-04-01'),
       (2, 20, '2023-04-02'),
       (3, 30, '2023-04-03'), ....; --limit 1.000

insert into transactions (id, value, date)
values (1001, 50, '2023-04-01'),
       (1002, 60, '2023-04-02'),
       (1003, 70, '2023-04-03'), ....; --limit 1.000

Result

quantity	method	time seconds	performance
1.000	Insert1	1,44s	-
1.000	Insert2	0,09s	16x
1.000	Insert3	0,09s	16x
10.000	Insert1	14,25s	-
10.000	Insert2	0,88s	16x
10.000	Insert3	0,83s	17x
100.000	Insert1	(2m 38s) 158,41s	-
100.000	Insert2	error	error
100.000	Insert3	8,25s	19x
1.000.000	Insert1	(31m 52s) 1952,32s	-
1.000.000	Insert2	error	error
1.000.000	Insert3	(1m 35s) 95s	20x

Update

Grupo 1:

Atualizar um campo ou mais, mas o valor é o mesmo para um determinado grupo:

Update1Job

update transactions
set date=now(),
    value=1
where id = 1;

update transactions
set date=now(),
    value=1
where id = 2;

update transactions
set date=now(),
    value=1
where id = 3;
...

Update2Job

update transactions
set date=now(),
    value=1
where id in (1, 2, 3, . . .);

Result

quantity	method	time seconds	performance
1.000	UpdateMassive1Job	1,00s	-
1.000	UpdateMassive2Job	0,01s	100x
10.000	UpdateMassive1Job	14,00s	-
10.000	UpdateMassive2Job	0,14s	100x
100.000	UpdateMassive1Job	(2m 36s) 156,00s	-
100.000	UpdateMassive2Job	1,00s	156x
1.000.000	UpdateMassive1Job	(26m 10s) 1570,00s	-
1.000.000	UpdateMassive2Job	15,00s	104x

Grupo 2:

Atualizar um campo ou mais, mas o valor é o diferente para cada registro:

Update3Job

update transactions
set date='2023-04-01',
    value=10
where id = 1;

update transactions
set date='2023-04-02',
    value=20
where id = 2;

update transactions
set date='2023-04-03',
    value=30
where id = 3;

Update4Job

update transactions
set date = case id when 1 then '2023-04-01'::date
     when 2 then '2023-04-02'::date 
     when 3 then '2023-04-03'::date
end
,
    value = case id when 1 then 10
     when 2 then 20 
     when 3 then 30
end
where id in (1, 2, 3);

Update5Job

update transactions as t
set date  = v.date,
    value = v.value from (
      values (1, '2023-04-01'::date, 10),
             (2, '2023-04-02'::date, 20),
             (3, '2023-04-03'::date, 30)
    ) as v(id, date, value)
where t.id = v.id;

Result

quantity	method	time seconds	performance
1.000	UpdateMassive3Job	1,00s	-
1.000	UpdateMassive4Job	0,08s	12x
1.000	UpdateMassive5Job	0,06s	16x
10.000	UpdateMassive3Job	14,00s	-
10.000	UpdateMassive4Job	0,73s	19x
10.000	UpdateMassive5Job	0,59s	23x
100.000	UpdateMassive3Job	(2m 44s) 164,00s	-
100.000	UpdateMassive4Job	8,00s	20x
100.000	UpdateMassive5Job	7,00s	23x
1.000.000	UpdateMassive3Job	(27m 10s) 1630,00s	-
1.000.000	UpdateMassive4Job	(1m 15s) 75,00s	21x
1.000.000	UpdateMassive5Job	58,00s	28x