?>Array ( [id] => 63 ) Performance & Automation: Daily Data Ingestion Folders - WeAreCAS
table addCaslibSubdir

Performance & Automation: Daily Data Ingestion Folders

Scénario de test & Cas d'usage

Geschäftskontext

An automated ETL process needs to create a unique subdirectory for each day's data load within an 'ingestion' caslib. This scenario tests the action's performance and reliability when called in rapid succession within a scripted, automated workflow.
Datenaufbereitung

Define the ingestion caslib and a SAS macro to generate a list of directory names for batch creation. This simulates a real-world automation script.

Kopiert!
1PROC CAS;
2 caslib ingestionZone path='/cas/data/ingest' dataSource={srcType='path'};
3RUN;
4%macro create_daily_folders(caslib_name, num_days);
5 %DO i = 1 %to &num_days;
6 %let folder_name = %sysfunc(intnx(day, %sysfunc(today()), -&i), yymmddn8.);
7 PROC CAS;
8 TABLE.addCaslibSubdir /
9 lib="&caslib_name",
10 path="&folder_name",
11 perms='PRIVATE';
12 RUN;
13 %END;
14%mend create_daily_folders;

Étapes de réalisation

1
Execute the macro to create 50 subdirectories, simulating 50 consecutive days of data loads. This tests performance and the use of parameter aliases ('lib' for 'caslib', 'perms' for 'permission').
Kopiert!
1%create_daily_folders(ingestionZone, 50);
2
After the batch creation, verify that all 50 directories were created successfully by listing the caslib contents and counting the results.
Kopiert!
1PROC CAS;
2 TABLE.fileInfo RESULT=r / caslib='ingestionZone';
3RUN;
4PROC PRINT DATA=r.FileInfo; where FileType='D'; RUN;
5PROC sql noprint;
6 select count(*) into :dir_count from r.FileInfo where FileType='D';
7QUIT;
8%put NOTE: Found &dir_count directories.;

Erwartetes Ergebnis


The macro should execute without errors, rapidly creating 50 distinct subdirectories named by date (e.g., '20251124', '20251123', etc.) within the '/cas/data/ingest' path. The final verification step should print a list of directories, and the SAS log should display 'NOTE: Found 50 directories.', confirming that the action performed correctly and efficiently in a batch automation context.