Big Data Management: Difference between revisions
From Stat DS Computing Wiki
Created page with "As your research experiments scale up in size, effective data management and storage becomes just as important as computational efficiency. This page will cover basic procedur..." |
No edit summary |
||
| Line 1: | Line 1: | ||
As your research experiments scale up in size, effective data management and storage becomes just as important as computational efficiency. This page will cover basic procedures and tools for moving, reading/writing, and storing large amounts of data. | As your research experiments scale up in size, effective data management and storage becomes just as important as computational efficiency. This page will cover basic procedures and tools for moving, reading/writing, and storing large amounts of data. | ||
Back to [[Beginner's Guide to Computation in Statistical Research]] | |||
== Interacting with Big Data on Remote Servers == | == Interacting with Big Data on Remote Servers == | ||
Revision as of 07:18, 10 April 2026
As your research experiments scale up in size, effective data management and storage becomes just as important as computational efficiency. This page will cover basic procedures and tools for moving, reading/writing, and storing large amounts of data.
Back to Beginner's Guide to Computation in Statistical Research
Interacting with Big Data on Remote Servers
| Title | Description | Technical Level | Relevance |
|---|---|---|---|
| CLI for data management (TBW) | Introduces basic Linux commands for moving data to/from a remote server | ⭐⭐⚫⚫⚫ | All server workflows |
Tools
| Title | Description | Technical Level | Relevance |
|---|---|---|---|
| Globus | A graphical application for moving and sharing large files within and between insitutions | ⭐⚫⚫⚫⚫ | All server workflows |
