Create an account

Very important

  • To access the important data of the forums, you must be active in each forum and especially in the leaks and database leaks section, send data and after sending the data and activity, data and important content will be opened and visible for you.
  • You will only see chat messages from people who are at or below your level.
  • More than 500,000 database leaks and millions of account leaks are waiting for you, so access and view with more activity.
  • Many important data are inactive and inaccessible for you, so open them with activity. (This will be done automatically)


Thread Rating:
  • 352 Vote(s) - 3.48 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Read lines synchronously from file in Node.js

#1
I need to parse a file line by line in the following format with Node.js:

13
13
0 5
4 3
0 1
9 12
6 4
5 4
0 2
11 12
9 10
0 6
7 8
9 11
5 3

It represents a graph. The first two lines are the number of edges and vertexes, followed by the edges.

I can accomplish the task with something like:

var fs = require('fs');
var readline = require('readline');
var read_stream = fs.createReadStream(filename);
var rl = readline.createInterface({
input: read_stream
});
var c = 0;
var vertexes_number;
var edges_number;
var edges = [];
rl.on('line', function(line){
if (c==0) {
vertexes_number = parseInt(line);
} else if (c==1) {
edges_number = parseInt(line);
} else {
edges.push(line.split(' '));
}
c++;
})
.on('end', function(){
rl.close();
})

I understand this kind of things might not be what Node.js was thought for, but the cascaded `if` in the `line` callback does not really look elegant / readable to me.

Is there a way to read synchronously lines from a stream like in every other programming language?

I'm open to use plugins if there is not a built-in solution.

[EDIT]

Sorry, I should have made clearer that **I would like to avoid loading the whole file in memory beforehand**
Reply

#2
My usual code part for such simple tasks:

var lines = require('fs').readFileSync(filename, 'utf-8')
.split('\n')
.filter(Boolean);

`lines` is an array of strings without empty ones.
Reply

#3
Personally, I like to use [`event-stream`](

[To see links please register here]

) to deal with streams. It's not necessary here but I used it for the code sample. It's simple, I parse to int and put everything inside `edges`, then when the file reading is done, I take the first element wich is `vertexes_number`, the new first element is `edges_number`

var fs = require('fs');
var es = require('event-stream');

var filename = 'parse-file.txt';

var vertexes_number, edges_number;
var edges = [];

fs.createReadStream(filename)
.pipe(es.split()) // split by lines
.pipe(es.map(function (line, next) {
// split and convert all to numbers
edges.push(line.split(' ').map((n) => +n));

next(null, line);
})).pipe(es.wait(function (err, body) {
// the first element is an array containing vertexes_number
vertexes_number = edges.shift().pop();

// the following element is an array containing edges_number
edges_number = edges.shift().pop();

console.log('done');
console.log('vertexes_number: ' + vertexes_number);
console.log('edges_number: ' + edges_number);
console.log('edges: ' + JSON.stringify(edges, null, 3));
}));
Reply

#4
This project on github.com does exactly what I needed:

[To see links please register here]


var readlines = require('n-readlines');
var liner = new readlines(filename);

var vertexes_number = parseInt(liner.next().toString('ascii'));
var edges_number = parseInt(liner.next().toString('ascii'));
var edges = [];
var next;
while (next = liner.next()) {
edges.push(next.toString('ascii').split(' '));
}
Reply

#5
Why not read them all into an array and then take out the first two elements with splice. I assume that your example is much simplified or else you would just read the whole file into memory and split it. If your actual case stores multiple graphs and you want to do something when each one is loaded for instance, you can put a test in your line event

var fs = require('fs');
var readline = require('readline');
var read_stream = fs.createReadStream(filename);
var rl = readline.createInterface({
input: read_stream
});

var buffer = [];

rl.on('line', function(line){
buffer.push(line.split(' '));
//Not sure what your actual requirement is but if you want to do
//something like display a graph once one has loaded
//obviously need to be able to determine when one has completed loading
if ( buffer.length == GRAPHLENGTH) { //or some other test
displayGraph(buffer);
buffer = [];
}
})
.on('close', function(){
//or do it here if there is only one graph
//displayGraph(buffer);
rl.close();
})

function displayGraph(buffer){
var vertexes_number = parseInt(buffer.splice(0,1));
var edges_number = parseInt(buffer.splice(0,1));
var edges = buffer;

//doYourThing(vertexes_number, edges_number, edges);
}
Reply



Forum Jump:


Users browsing this thread:
1 Guest(s)

©0Day  2016 - 2023 | All Rights Reserved.  Made with    for the community. Connected through