C# Efficient text file line deletion method

When processing text files, it is often necessary to delete specific lines. This article explores efficient solutions to implement this task in C#, especially for handling large text files.

Solution Overview

This scheme creates a temporary file and selectively copy the lines from the original file into the temporary file, excluding the lines to be deleted. After processing the target line, the temporary file replaces the original file, thereby deleting unnecessary lines.

Code implementation

string tempFile = Path.GetTempFileName();

using (StreamReader sr = new StreamReader("file.txt"))
using (StreamWriter sw = new StreamWriter(tempFile))
{
    string line;

    while ((line = sr.ReadLine()) != null)
    {
        if (line != "removeme")
            sw.WriteLine(line);
    }
}

File.Delete("file.txt");
File.Move(tempFile, "file.txt");

This method ensures that only necessary lines are retained while excluding target lines from updated files.

Other considerations

• Performance: For large files, this method minimizes memory usage and disk I/O by processing lines sequentially.
• Encoding: The code assumes that the text file is encoded using UTF-8. Adjust the encoding as needed.
• Memory processing: For smaller files, memory processing may be more efficient. Please consider the following:

File.WriteAllLines("file.txt",
    File.ReadLines("file.txt").Where(l => l != "removeme").ToList());

This method avoids creating temporary files, but needs to be executed immediately using ToList().