Roadmaps / Data / Setup and lab safety
Read first
Setup and lab safety
You need one laptop with 16 GB of RAM if possible (8 GB works with the smaller scope), 30 GB of free disk, Docker, Python, DuckDB and a container each for Postgres and Kafka. Everything is free. Spark and Flink run in local mode.
| Risk | Rule |
|---|---|
| Real personal data | Use generated data only. Never load real customer data into a practice lake. The deletion exercise uses fake users. |
| Backfills overwrite tables | Run backfills on a copy or a branch of the table first. Keep a snapshot you can roll back to. |
| Cloud storage costs | The core scope needs no cloud account. If you try one, set a budget with a hard cap, and delete the buckets after. |
| Credentials in pipelines | Keep passwords in environment variables or a secrets file outside Git. Never commit a connection string. |