🧵GitLab reduced downtime by 61% by migrating its cloud platform. Kakao, on the other hand, was completely unable to migrate.
In short: In 2022, 32,000 servers in one building went down, leaving 53 million accounts inaccessible.
Naver next door restored its service in just a few hours using a separate mirror.
Filecoin Onchain Cloud provides this portability with zero outbound traffic.
1/ GitLab's Improvement
GitLab's platform availability improved from 99.68% to 99.88% after migrating from Microsoft Azure to Google Cloud in 2018.
According to GitLab's engineering blog (which details the entire migration process), weekly downtime was reduced by 61%.
2/ Migration Mechanism
This migration involved five months of preparation and 11 hours of maintenance and switchover by 15 to 25 core infrastructure engineers.
The GitLab team reported that weekly downtime decreased from 32 minutes to 13 minutes, resulting in a 61% improvement in overall availability.
3/ Kakao's Dilemma
32,000 servers were located in a data center in Pangyo, South Korea, operated by SK C&C for Kakao.
On October 15, 2022, a battery fire caused a mains grid failure, instantly disrupting all Kakao services and affecting over 53 million active accounts nationwide.
4/ Who Said It?
South Korean President Yun Seok-youl described the blackout as a "paralysis of the national communications network."
He stated, "This paralyzed the operation of the country," and described the extent of dependence quietly created nationwide by an overloaded data center.
5/ Next Door
Naver, a competitor located in the same burning building as Kakao, restored full service within hours, while Kakao took several days.
The difference lies in the fact that Filecoin Onchain Cloud runs in a separate, self-owned data center, employing dual-active mirroring—a redundancy mechanism never before present in Kakao's architecture.
6/ The Key is
GitLab migrated because its data uses a portable, open format.
Filecoin Onchain Cloud is built in the same way, compatible with S3 and using content addressing, and does not charge any egress fees. Therefore, the portability that Kakao never had is provided by default, eliminating the need for an emergency rebuild.
Maintaining data portability across different providers:
450
0
0
38
Filecoin TL;DR
07-28 20:19
フォロー
🧵WM Technology Pays $2.3 Million for Unused AWS Capacity
In short: AWS charged $53,791 for moving 1 PB of data out of storage.
Datadog, Snowflake, and others exposed this practice. WM Technology paid a $2.3 million penalty.
Filecoin Onchain Cloud charges $2.50 per TiB per month with $0 outbound traffic fees and no opt-out clause.
1/ Cost
The $53,791 is the computational cost of moving 1 PB of data out of standard AWS S3 storage using AWS's published tiered outbound traffic rates.
Google Cloud charges $80,298 for the same transfer, and Azure follows closely at $53,681, both using their published rate tables.
2/ Datadog
Datadog disclosed in its filing with the U.S. Securities and Exchange Commission (SEC) that it committed to investing $225 million in AWS and $37.5 million in Google Cloud over three years.
The filing explicitly states that neither party may terminate the AWS or GCP agreement for its own convenience within the three-year period, without exception.
3/ Snowflake
Snowflake disclosed in its filing with the SEC that its total cloud service commitments on AWS, Azure, and Google Cloud amount to billions of dollars, valid until May 2028.
Migrting customer data to bare metal will trigger payment for the unfulfilled portion of the contract.
4/ Who Predicted This?
Lina Khan raised this issue during a panel discussion on cloud service concentration at the Council on Foreign Relations while serving as chair of the Federal Trade Commission (FTC).
She noted that this lock-in effect would impose some contractual restrictions and described what regulators found when reviewing cloud contracts.
5/ WM Technology
WM Technology, the parent company of Weedmaps, disclosed in its financial filings that it reduced its AWS usage to below the contractual minimum in fiscal year 2025, and consequently set aside $2.3 million in loss reserves.
It directly paid AWS for previously optimized capacity.
6/ End
Data ownership depends on a company's ability to transfer data if the service provider, market, or contract changes, not just on the current account holder.
Filecoin Onchain Cloud operates at $2.50 per TiB per month per replica. There are no exit fees or negotiated exit terms.
Cancellation of exit fees and contract penalties:
374
0
0
45
Filecoin TL;DR
07-27 16:36
フォロー
We are pleased to announce the successful conclusion of the Filecoin TLDR Builder Challenges Round 3 (hosted by @LoopsHouse)!
This round received 63 entries, with 14 developers submitting projects exploring various Filecoin tools—all designed to make it easier for agents, developers, and users to interact with Filecoin.
View the winners! 👇
A heartfelt thank you to all the developers, experimenters, and submitters who participated in Round 3! Click here to view all entries on @LoopsHouse:
Stay tuned for the upcoming Filecoin TLDR Builder Challenges Round 4!
491
0
0
40
Filecoin TL;DR
07-24 17:25
フォロー
⏰ Reminder: Filecoin TLDR Build Challenge Round 3 – Submission Deadline Approaching!
📅 The build cycle ends on July 26th!
Don't miss your chance – be sure to submit your Filecoin tool before the deadline.
You will need:
- A working demo or prototype
- Actual interaction with the Filecoin technology stack
- A platform to publicly showcase your build
- Post a public thread on X and tag @Filecoin and @FilecoinTLDR
Keep it up – we can't wait to see your work! 🚀
Full details and submission link:
395
0
0
23
Filecoin TL;DR
07-24 14:55
フォロー
🧵Checkpoints Saved Meta Llama 3 Training
In short: Meta's Llama 3 model experienced 419 outages over 54 days, saving 5TB of checkpoints every few hours to protect $63.7 million worth of training.
Filecoin Onchain Cloud prices this long-term storage solution at $2.50/TiB per replica per month.
1/ Outages
Over 54 days, Meta's Llama 3 model (405B pre-trained) experienced 419 unexpected outages on a cluster of 16,384 Nvidia H100 GPUs, averaging one every three hours.
Meta noted in its July 2024 technical report that most outages were hardware-related, with the remainder attributed to software and infrastructure.
2/ Checkpoint Frequency
Each checkpoint for the model was close to 5TB, and the cluster saved it every 2 to 4 hours.
At such a massive scale, outages are inevitable, not rare exceptions, which is why the continuous 54-day save never stopped.
3/ Meta's Improvements
Meta's engineering team directly described the fixes in their Llama 3 announcement.
They specifically built a new scalable storage system to reduce the overhead of checkpointing and rollback, and improved the detection of quiescent data corruption within the cluster.
4/ Risks
Based on Meta's own published GPU hours, the full 54-day run cost at standard H100 cloud rates was $63.7 million.
If this data were lost due to an unrecoverable crash, the loss could be between $15 million and $30 million, and these losses would be irreversible.
5/ Why It Continues to Work After the Run Ends
Checkpointing remains crucial after training ends.
Downstream teams use a specific amount of intermediate save data to fine-tune model variants, run security assessments, and accurately demonstrate the model's state when specific functionalities occur.
6/ FOC
Filecoin Onchain Cloud is designed specifically for this type of long-term data storage, priced at $2.50 per TiB per month per replica.
Two replicas are provided by default to store records that need to remain searchable and verifiable after training.
Storage checkpoints eliminate concerns about costs increasing over time:
490
0
0
34
Filecoin TL;DR
07-23 03:58
フォロー
Filecoin Statistics:
Network Quality Adjustment Power: 13.73 EiB
24-Hour Block Reward: 60,895.98 FIL
Total Block Reward: 404,884,574.16 FIL
Reward per Success: 4.31 FIL
Average Number of Blocks per TipSet: 4.88
Average Number of Messages per TipSet: 49.07
Number of Active Nodes: 565
FIL Burned: 42,641,177.68 FIL
🧵The Cost of Leaving
In short: Even with an exemption, migrating a complete 1 PB of cloud data will incur a transfer fee of $80,000 to $85,000, plus a $36,000 API fee for 100 billion read requests.
Filecoin Onchain Cloud costs $2.50 per TB per month to operate, with no outbound traffic fees and no time limit for exiting.
1/ Conditions
AWS, Google Cloud, and Azure now offer waivers of outbound traffic fees for customers completely leaving their platforms, in response to EU data protection laws.
All three require migrating all workloads and terminating the account, and do not offer partial exit services.
2/ Time Limits
Google Cloud's exemption policy has time limits. First, there is a 14-day review period, followed by a 30-day migration completion deadline.
Missing this window will result in the exemption being revoked, and the fee-free exit will be arranged by the service provider.
3/ Hidden Costs
Even with waived outbound traffic fees, request costs still exist.
Retrieving 1PB of data stored as 10KB files would require approximately 100 billion API read requests. At Google Cloud's rate of $0.36 per million Class B operations, this alone amounts to $36,000.
4/ Exit Costs
A three-year model comparison shows that on a standard hyperscale cloud platform, using published outbound traffic rates, fully migrating 1PB of data would cost between $80,000 and $85,000 in network transfer costs alone, not including labor costs.
Outbound traffic itself accounts for 40% to 42% of total workload costs.
5/ Who Proposed This Viewpoint
Kelsey Hightower, a Kubernetes expert and former Google Cloud engineer, raised this point in an industry presentation in 2024. He stated that data egress fees are killing businesses.
In the cloud storage space, the default architecture almost everywhere remains free data ingress coupled with pay-as-you-go egress traffic.
6/ Conclusion
Filecoin Onchain Cloud employs a completely different architecture. Storage costs $2.50 per TiB per month per replica, with two replicas included standard, and there are no egress fees.
You don't need to meet any conditions, close any accounts, or leave before a deadline.
Your data should not incur any additional costs. Start using Filecoin Onchain Cloud today:
445
0
0
29
Filecoin TL;DR
07-21 01:28
フォロー
Filecoin Statistics:
Network Quality Adjusted Hashrate: 13.85 EiB
Block Rewards (24 hours): 62,281 FIL
Total Block Rewards: 404.8 million FIL
24-Hour Gas Fees: 89 FIL
Burns: 42.6 million FIL
Single Win Reward: 4.31 FIL
Average Number of Blocks per TipSet: 4.96
Average Number of Messages per TipSet: 46.9
Active Nodes: 572
Data Capacity: 1.37 EiB
366
0
0
23
Filecoin TL;DR
07-16 00:15
フォロー
Filecoin Statistics:
Contract Addresses: 801,549
Verified Contracts: 127
Total Transaction Volume: 9,931,298
24-Hour Transaction Volume: +6,101
Contract Balance: 15,889,510 FIL
Committed Capacity: 199.03 PiB
Data Capacity: 1.39 EiB
24-Hour Gas Fee: 52.52 FIL
Active Contracts: 4,822
Active Nodes: 573