Sys.setLanguage("pt_BR") or
Sys.setLanguage("en") switches the language within a session.
Previously the messages mixed both languages and the shipped translations
were never applied, because the cli calls did not go through gettext().R-pt_BR.po; R-en.po was
removed because the source language is English.{...} code blocks of the original
message unchanged.cli, which then always used the singular.read_only = TRUE when an instance for the
same path is already open (read_only can't be applied to the database instance ... which already exists):
caged_load() and caged_adjustments_load() now close their write
connection before calling caged_info()..disconnect() helper now also calls
duckdb::duckdb_shutdown(), since dbDisconnect(shutdown = TRUE) no
longer releases the cached instance.caged_info(), caged_to_parquet() and caged_update() reuse an
already-open instance (e.g. a user connection) instead of failing.caged_for and caged_exc: the deduplication key
competencia was taken from competenciamov, but FOR/EXC files contain
movements from earlier months that repeat across files, so loading month
t+1 silently skipped rows whose competenciamov had already appeared
(e.g. 46% of CAGEDFOR202303). The key is now competenciadec for
caged_for and competenciaexc for caged_exc (one value per file).
caged_mov, caged_antigo and caged_ajustes are unchanged.
Action required: databases built with earlier versions should drop
caged_for and caged_exc and reload them with caged_load(). Use
competenciamov for analyses by month of movement.C/POSIX): accented headers such as
competênciamov became compet_nciamov, and caged_to_duckdb() aborted
with "Nenhuma coluna de competencia encontrada". .normalize_names() now
uses chartr() and read_delim() uses name_repair = "unique_quiet".caged_download_layouts() now downloads the layout files from the
HuggingFace repository instead of the MTE FTP server, which was slow and
often unreachable. Local file names have accents removed; the original
name is kept in the arquivo column.caged_update() now compares competencies table by table instead of using
the maximum competency. It also fills gaps inside the period already in the
database (e.g. a FOR file whose download failed) and calls the pipeline
once per year, so it no longer downloads unneeded months (requesting
202512 and 202601 used to also download 202501 and 202612).caged_load(), caged_download()) and CAGED Adjustments from
2002 (caged_adjustments_load()). Earlier years now fail immediately with
a clear message instead of finding no files.caged_to_duckdb(overwrite_competencies = TRUE) now runs DELETE and INSERT
in a single transaction: if the insert fails, the previous data is kept.caged_to_parquet(): partition_by accepts several columns, overwrite = FALSE is respected for partitioned output, a missing partition column gives
a clear warning, and paths containing ' work. Partition directories for
code columns (competencies, uf, regiao, municipio) are now named
competencia=202301/uf=35 instead of competencia=202301.0/uf=35.0; the
cast to integer happens only in the export, so the database schema is
unchanged.caged_adjustments_load() for 2002-2009, where the source is one annual
file per year: the file is now downloaded and parsed once instead of once
per requested month (about 6x faster for a full year), and only the
requested months are stored (previously the whole year was stored)..part file and renamed when
complete, so an interrupted write can no longer leave a truncated file
that the cache would treat as valid.caged_hf_files(verbose = FALSE) no longer prints the "Querying
HuggingFace" message.skip_on_cran() and skip_if_offline().on.exit() without add = TRUE
after local_mocked_bindings() discarded the restore handler.caged_to_duckdb() documentation now describes the actual deduplication
mechanism and the key column per table (it never used INSERT OR IGNORE).?datacaged now lists caged_update(), caged_to_parquet() and
caged_download_layouts().inst/CITATION now reads the version from DESCRIPTION.caged_info() sample output in the vignettes.caged_ftp_files() removed — use caged_hf_files() instead.caged_hf_files(type = "ajustes") returning years > 2019 — now
correctly filters to years <= 2019.CAGEDEST_AJUSTES_ was
incorrectly read as CAGEDAJUSTES_); updated prefixo_len from 13 to 17.CAGED_AJUSTES/2002a2009/ folder (annual files
CAGEDEST_AJUSTES_AAAA.7z)..insert_rows() to handle schema evolution when loading data from
multiple years with different column sets (ALTER TABLE ... ADD COLUMN).\donttest{} examples that require network access or a populated
database to \dontrun{} to pass R CMD check --run-donttest.caged_ftp_files alias in documentation.concurrency, CI=true, reduced coverage
timeout to 15 min, added dependabot.yml.~/Downloads/, D:/dados/)
with portable alternatives (file.path(tempdir(), ...), file.path("~", ...)).ftp://ftp.mtps.gov.br/pdet/microdados/) to
HuggingFace dataset (https://huggingface.co/datasets/alexsandroprado/caged).
Downloads now use HTTPS — no more FTP port 21 issues behind firewalls.caged_hf_files(): replaces caged_ftp_files() — lists available competencies
from the HuggingFace repository via its REST API.caged_update(): incremental update — downloads only competencies not yet in
the local DuckDB bank, querying the bank max competência and HuggingFace availability.caged_to_parquet(): exports DuckDB tables to Parquet files using DuckDB's native
COPY TO ... (FORMAT PARQUET) — supports optional column partitioning. — lists available competencies
from the HuggingFace repository via its REST API.caged_ftp_files(): kept as an alias with a deprecation warning. Will be removed
in a future version. Use caged_hf_files() instead.download.R: .ftp_url() renamed to .hf_url(); .download_file() now uses
method = "libcurl" with HTTPS and --location to follow CDN redirects.availability.R: .ftp_list() renamed to .hf_list_dir(); uses httr2 instead
of curl FTP listing.status.R: caged_status() now checks HuggingFace API endpoint.httr2 (>= 1.0.0) to Imports.DESCRIPTION: URL do FTP do MTE adicionada ao campo Description:
conforme política do CRAN.man/*.Rd: \dontrun{} substituído por \donttest{} em todos os
exemplos — os exemplos requerem internet mas podem ser executados
pelo usuário.man/*.Rd: db_path = "caged.duckdb" substituído por
db_path = file.path(tempdir(), "caged.duckdb") em todos os exemplos
para evitar escrita no diretório de trabalho do usuário.DESCRIPTION: 'DuckDB' entre aspas simples em Title e Description
conforme política do CRAN.Primeira versão do pacote — submetida ao CRAN.
caged_load() — pipeline completo: download → parse → DuckDB (Novo CAGED e antigo).caged_download() — baixa arquivos .7z do FTP do MTE com cache local e retry automático.caged_parse() — lê e normaliza microdados de um arquivo .7z.caged_parse_batch() — versão vetorizada de caged_parse() com barra de progresso.caged_to_duckdb() — grava dados no DuckDB com controle de duplicatas por competência.caged_connect() — abre conexão DBI com o banco DuckDB.caged_info() — exibe estatísticas das tabelas no banco.caged_adjustments_load() — pipeline completo para o CAGED Ajustes.caged_status() — verifica disponibilidade do servidor FTP do MTE.caged_ftp_files() — lista competências disponíveis no FTP.caged_download_layouts() — baixa os layouts oficiais do MTE.uf_codigos — tabela de UFs com códigos IBGE e regiões.