Multiple Files and Background Queues
Coordinate batch imports and move long-running spreadsheet jobs away from request-response traffic.
Updated
importDomainFiles()queue()pdoQueue()workQueue()What you will learn
- Import several files with one coordinator.
- Choose filesystem or PDO-backed queue storage.
- Run retryable and idempotent background jobs.
Import several files in one operation#
$result = MnbExcel::importDomainFiles(
'products',
$files,
$pdo,
'products',
[
'stop_on_error' => false,
'import_options' => [
'duplicate_strategy' => 'update',
'unique_by' => ['sku'],
],
]
);Use a filesystem queue on one host#
$queue = MnbExcel::queue(__DIR__ . '/storage/queue');
$jobId = $queue->enqueueDomainImport(
'products',
'products.xlsx',
$databaseConfig,
'products'
);
$result = MnbExcel::workQueue(__DIR__ . '/storage/queue', [
'max_jobs' => 50,
'max_attempts' => 3,
]);This backend is simple and useful for one server with a shared filesystem.
Use a transactional PDO queue for multiple workers#
$queue = MnbExcel::pdoQueue($pdo, 'mnb_excel_queue');
$queue->enqueueDomainImport(
'orders',
'orders.xlsx',
$databaseConfig,
'orders'
);
// Run from a CLI worker process.
$queue->work([
'max_jobs' => 100,
'max_attempts' => 5,
'retry_delay_seconds' => 60,
]);Make jobs retry-safe#
- Use stable unique keys and idempotent duplicate strategies.
- Store uploads in a durable path before enqueueing.
- Do not put database passwords or workbook passwords into public logs.
- Record job attempts, final status, counts, output paths, and safe error messages.
- Move permanently failed jobs to a review queue rather than retrying forever.
Was this guide useful?Use GitHub issues for corrections, missing examples, or unclear behavior.
Open an issue