Base64 encode/decode large file

作者: 江大鱼 发表于 2008-04-20 21:48 原文链接 阅读: 782 评论: 0

The class System.Convert provide two basic methods "ToBase64String()" and "Convert.FromBase64String()" to encode a byte array to a base64 string and decode a base64 string to a byte array. public   string  Encode( byte [] data) {     return Convert.ToBase64String(data); }          public   byte [] Decode( string  strBase64) {     return Convert.FromBase64String(strBase64); } It is very good to use them to encode and decode base64. But in some case, it is a disaster. For example, if you want to encode a 4 gb file to base64, the code above must throw an OutOfMemory exception., because you must read the file into a byte array. So we need to look for another way to encode and decode by base64. Long days ago, a man have posted an article about how to deal with it. http://blogs.microsoft.co.il/blogs/kim/archive/2007/10/09/base64-encode-large-files-very-large-files.aspx This man use XmlWriter to work around it. By researching the basis of the Base64 encoding in rfc, I found another more directly way to deal with it. According rfc3548, base64 encode data in the unit of 3 bytes to 4 bytes, if the last part's length is less than 3, the char '=' will be padded. So we can encode file in small chunks whose size is 3, then we can get the encoding data of the file by combiling encoding data of every chunks. So I have below code: public   void  EncodeFile( string  inputFile,  string  outputFile) {             using(FileStream inputStream = File.Open(inputFile, FileMode.Open, FileAccess.Read, FileShare.Read))             {                 using(StreamWriter outputWriter = new StreamWriter(outputFile, false, Encoding.ASCII))                 {                     byte[] data = new byte[3 * 1024]; //Chunk size is 3k                     int read    = inputStream.Read(data, 0, data.Length);                                          while(read > 0)                     {                         outputWriter.Write(Convert.ToBase64String(data, 0, read));                         read = inputStream.Read(data, 0, data.Length);                     }                                          outputWriter.Close();                                     }                                  inputStream.Close();             }         }          public   void  DecodeFile( string  inputFile,  string  outputFile)          {             using (FileStream inputStream = File.Open(inputFile, FileMode.Open, FileAccess.Read, FileShare.Read))             {                 using (FileStream outputStream = File.Create(outputFile))                 {                     byte[] data = new byte[4 * 1024]; //Chunk size is 4k                     int read = inputStream.Read(data, 0, data.Length);                     byte[] chunk    = new byte[3 * 1024];                                  while (read > 0)                     {                         chunk = Convert.FromBase64String(Encoding.ASCII.GetString(data, 0, read));                         outputStream.Write(chunk, 0, chunk.Length);                         read = inputStream.Read(data, 0, data.Length);                     }                     outputStream.Close();                 }                 inputStream.Close();             }         } These methods also can be improved to support mime format (76 chars per line). public   static   void  EncodeFile( string  inputFile,  string  outputFile)          {             using(FileStream inputStream = File.Open(inputFile, FileMode.Open, FileAccess.Read, FileShare.Read))             {                 using(StreamWriter outputWriter = new StreamWriter(outputFile, false, Encoding.ASCII))                 {                     byte[] data = new byte[57 * 1024]; //Chunk size is 57k                     int read    = inputStream.Read(data, 0, data.Length);                                          while(read > 0)                     {                         outputWriter.WriteLine(Convert.ToBase64String(data, 0, read, Base64FormattingOptions.InsertLineBreaks));                         read = inputStream.Read(data, 0, data.Length);                     }                                          outputWriter.Close();                                     }                                  inputStream.Close();             }         }          public   static   void  DecodeFile( string  inputFile,  string  outputFile)          {             using (StreamReader reader = new StreamReader(inputFile, Encoding.ASCII, true))             {                 using (FileStream outputStream = File.Create(outputFile))                 {                                     string line = reader.ReadLine();                     while (!string.IsNullOrEmpty(line))                     {                         if (line.Length > 76)                             throw new InvalidDataException("Invalid mime-format base64 file");                         byte[] chunk = Convert.FromBase64String(line);                         outputStream.Write(chunk, 0, chunk.Length);                         line = reader.ReadLine();                     }                     outputStream.Close();                 }                 reader.Close();             }         }

好久没更新BLOG了， 今天来篇英文的文章，真是倍感吃力阿。

作者： 江大鱼
出处： http://jzywh.cnblogs.com
本文版权归作者和博客园共有，欢迎转载，但未经作者同意必须保留此段声明，且在文章页面明显位置给出原文连接，否则保留追究法律责任的权利。

评论: 0　查看评论　发表评论

百度期待您的加盟

最新新闻：
· 分析：雅虎与谷歌新闻战略之争(2010-07-11 15:14)
· 百度今日赴硅谷为框计算招聘顶级工程师(2010-07-11 15:08)
· 团购网站的末日(2010-07-11 14:19)
· 纸和笔的小众需求(2010-07-11 13:45)
· 涂雅速涂：黑客在行动，唐峻很郁闷(2010-07-11 13:07)

编辑推荐：Swifter C#之inline还是不inline，这是个问题

网站导航：博客园首页  个人主页  新闻  闪存  小组  博问  社区  知识库