]
03-12-2019, 07:49 PM
Introduction
You can follow along by reading the first tutorial about
Combo Lists
You can get a shit ton of combo lists from my old github repo (
We can read a list (the list im using has 8,000 lines) into a generic list:
In the code above we are reading a file named fileName into a generic string list named lines. Two libraries are needed for this: System.Collections.Generic and System.IO.
Removing Duplicates
When removing duplicates you can use
That's it, you have removed all the duplicates from that generic list. So now that we have remove the duplicates we can write it back out to a new file named "no_dupes". So let's put it all together.
Dealing with HUGE Combo Lists
When dealing with large combo lists the best thing to do is read a bit at a time. There is no reason to ever store that much in a single file but if you do (and you know who you are) we can use a temporary file and read a little bit at a time. This will keep your software from locking up and your computer free from dying. This is a 24mb flat file with 800k+ combos
So what do we do? Chunk it up into smaller sections.
So to use the method we would supply the name of the file, which line to start with and then how many to return from the line we started at, for example:
We are starting at line 50 in the file and returning 100 lines.
Conclusion
I am stopping here and going to provide you with the class I've written so far. I changed the ReadLargeCombo method to return the array of lines instead of printing them out on the spot (since that would be bad OOP practice). If you have suggestions for the next tutorials, let me know. If you have problems with your combo lists and would like to have a solution to the problem, let me know, I will add it to the next tutorial.
You can follow along by reading the first tutorial about
[To see links please register here]
. If you have already read that one or know how to do it, please feel free to continue. You can read my other[To see links please register here]
. All of my tutorials are written under Linux, and I will not pander to the Windows kiddies. I have been developing software since before .NET was around and normally don't even use an IDE, I either use[To see links please register here]
,[To see links please register here]
, or[To see links please register here]
. Nothing wrong with using a full IDE, especially if you're new.Combo Lists
You can get a shit ton of combo lists from my old github repo (
[To see links please register here]
) that has 17 million combos. You can use any of those to practice your skills and use the code presented here. Most of the combo lists people are using are huge. When you are dealing with a large amount of data, reading line by line into a string[] array is just stupid, it will slow your computer down and probably crash your software. The lists in my repo shouldn't have duplicates in them and they should all be a colon delimited format.We can read a list (the list im using has 8,000 lines) into a generic list:
Hidden Content
In the code above we are reading a file named fileName into a generic string list named lines. Two libraries are needed for this: System.Collections.Generic and System.IO.
Removing Duplicates
When removing duplicates you can use
[To see links please register here]
since we are using a generic list to hold the combos. It requires a simple addition to one of our lines of code.Hidden Content
That's it, you have removed all the duplicates from that generic list. So now that we have remove the duplicates we can write it back out to a new file named "no_dupes". So let's put it all together.
Hidden Content
Dealing with HUGE Combo Lists
When dealing with large combo lists the best thing to do is read a bit at a time. There is no reason to ever store that much in a single file but if you do (and you know who you are) we can use a temporary file and read a little bit at a time. This will keep your software from locking up and your computer free from dying. This is a 24mb flat file with 800k+ combos
[To see links please register here]
. This is not something you should load into a string[] or even a generic list all at once.So what do we do? Chunk it up into smaller sections.
Hidden Content
So to use the method we would supply the name of the file, which line to start with and then how many to return from the line we started at, for example:
Hidden Content
We are starting at line 50 in the file and returning 100 lines.
Conclusion
I am stopping here and going to provide you with the class I've written so far. I changed the ReadLargeCombo method to return the array of lines instead of printing them out on the spot (since that would be bad OOP practice). If you have suggestions for the next tutorials, let me know. If you have problems with your combo lists and would like to have a solution to the problem, let me know, I will add it to the next tutorial.
Hidden Content
