Compare 2 files and print matching values to a new file
up vote
0
down vote
favorite
I have a file which contains 12 Million lines each line is broken into 8 parts all of which are separated by ";".
File1 Example:
AAA;BBB;CCC;DDD;EEE;FFF;GGGXX;HHH
Each line has a unique ID in $7.
I have another file (File2) which has ~20K of these unique IDs listed as:
GGGXX;
GGGXY;
GGGXZ;
Each on its own line.
I need the result to create a new file with the complete data row from File1 based on the IDs in File2.
I cannot seem to get it to work when File2 has more than 1 entry using the code below.
awk -F ";" 'NR==FNR{a[$1]} ($7 in a){print}' File2.txt File1.txt
Is there something wrong with how the IDs are listed in File2 or is there something else that I am doing wrong?
text-processing awk
asked Nov 30 at 18:37
Leo P
11
New contributor
Leo P is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
up vote
0
down vote
favorite
I have a file which contains 12 Million lines each line is broken into 8 parts all of which are separated by ";".
File1 Example:
AAA;BBB;CCC;DDD;EEE;FFF;GGGXX;HHH
Each line has a unique ID in $7.
I have another file (File2) which has ~20K of these unique IDs listed as:
GGGXX;
GGGXY;
GGGXZ;
Each on its own line.
I need the result to create a new file with the complete data row from File1 based on the IDs in File2.
I cannot seem to get it to work when File2 has more than 1 entry using the code below.
awk -F ";" 'NR==FNR{a[$1]} ($7 in a){print}' File2.txt File1.txt
Is there something wrong with how the IDs are listed in File2 or is there something else that I am doing wrong?
text-processing awk
asked Nov 30 at 18:37
Leo P
11
New contributor
Leo P is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Are you aware that'Fshould be-F? Is that just a typo?
– glenn jackman
Nov 30 at 19:34
This is not a bug, but it's a bit sloppy:a[$21++]-- that creates an array key for the value field 21 (the value for that array key is 0 or the empty string), then it increments the field 21 value. The++has no effect on the array at all and can be removed. You probably intendeda[$21]++, but it's not necessary that the array has a value for that key.
– glenn jackman
Nov 30 at 19:36
Having said that, you do not actually ask a question in your question. Please read How do I ask a good question?
– glenn jackman
Nov 30 at 19:38
@glennjackman Thanks for your help. I had made some typos. What I am trying to do is have both files compared and as I said in the post, File 1 has all of my data and File 2 $1 has the small group of data that I need to pull out of File 1 $7. When I try this and have File 2 with only 1 data point, it works, But when I have all 20K data points, I am not given any data from File1. I'll edit the main post with more information.
– Leo P
Nov 30 at 21:06
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I have a file which contains 12 Million lines each line is broken into 8 parts all of which are separated by ";".
File1 Example:
AAA;BBB;CCC;DDD;EEE;FFF;GGGXX;HHH
Each line has a unique ID in $7.
I have another file (File2) which has ~20K of these unique IDs listed as:
GGGXX;
GGGXY;
GGGXZ;
Each on its own line.
I need the result to create a new file with the complete data row from File1 based on the IDs in File2.
I cannot seem to get it to work when File2 has more than 1 entry using the code below.
awk -F ";" 'NR==FNR{a[$1]} ($7 in a){print}' File2.txt File1.txt
Is there something wrong with how the IDs are listed in File2 or is there something else that I am doing wrong?
text-processing awk
asked Nov 30 at 18:37
Leo P
11
New contributor
Leo P is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
I have a file which contains 12 Million lines each line is broken into 8 parts all of which are separated by ";".
File1 Example:
AAA;BBB;CCC;DDD;EEE;FFF;GGGXX;HHH
Each line has a unique ID in $7.
I have another file (File2) which has ~20K of these unique IDs listed as:
GGGXX;
GGGXY;
GGGXZ;
Each on its own line.
I need the result to create a new file with the complete data row from File1 based on the IDs in File2.
I cannot seem to get it to work when File2 has more than 1 entry using the code below.
awk -F ";" 'NR==FNR{a[$1]} ($7 in a){print}' File2.txt File1.txt
Is there something wrong with how the IDs are listed in File2 or is there something else that I am doing wrong?
text-processing awk
text-processing awk
asked Nov 30 at 18:37
Leo P
11
New contributor
Leo P is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Nov 30 at 18:37
Leo P
11
New contributor
Leo P is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited Nov 30 at 21:30
asked Nov 30 at 18:37
Leo P
11
New contributor
Leo P is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Nov 30 at 18:37
Leo P
11
asked Nov 30 at 18:37
Leo P
11
11
New contributor
Leo P is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Leo P is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Leo P is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Are you aware that'Fshould be-F? Is that just a typo?
– glenn jackman
Nov 30 at 19:34
This is not a bug, but it's a bit sloppy:a[$21++]-- that creates an array key for the value field 21 (the value for that array key is 0 or the empty string), then it increments the field 21 value. The++has no effect on the array at all and can be removed. You probably intendeda[$21]++, but it's not necessary that the array has a value for that key.
– glenn jackman
Nov 30 at 19:36
Having said that, you do not actually ask a question in your question. Please read How do I ask a good question?
– glenn jackman
Nov 30 at 19:38
@glennjackman Thanks for your help. I had made some typos. What I am trying to do is have both files compared and as I said in the post, File 1 has all of my data and File 2 $1 has the small group of data that I need to pull out of File 1 $7. When I try this and have File 2 with only 1 data point, it works, But when I have all 20K data points, I am not given any data from File1. I'll edit the main post with more information.
– Leo P
Nov 30 at 21:06
add a comment |
Are you aware that'Fshould be-F? Is that just a typo?
– glenn jackman
Nov 30 at 19:34
This is not a bug, but it's a bit sloppy:a[$21++]-- that creates an array key for the value field 21 (the value for that array key is 0 or the empty string), then it increments the field 21 value. The++has no effect on the array at all and can be removed. You probably intendeda[$21]++, but it's not necessary that the array has a value for that key.
– glenn jackman
Nov 30 at 19:36
Having said that, you do not actually ask a question in your question. Please read How do I ask a good question?
– glenn jackman
Nov 30 at 19:38
@glennjackman Thanks for your help. I had made some typos. What I am trying to do is have both files compared and as I said in the post, File 1 has all of my data and File 2 $1 has the small group of data that I need to pull out of File 1 $7. When I try this and have File 2 with only 1 data point, it works, But when I have all 20K data points, I am not given any data from File1. I'll edit the main post with more information.
– Leo P
Nov 30 at 21:06
Are you aware that
'F should be -F? Is that just a typo?– glenn jackman
Nov 30 at 19:34
Are you aware that
'F should be -F? Is that just a typo?– glenn jackman
Nov 30 at 19:34
This is not a bug, but it's a bit sloppy:
a[$21++] -- that creates an array key for the value field 21 (the value for that array key is 0 or the empty string), then it increments the field 21 value. The ++ has no effect on the array at all and can be removed. You probably intended a[$21]++, but it's not necessary that the array has a value for that key.– glenn jackman
Nov 30 at 19:36
This is not a bug, but it's a bit sloppy:
a[$21++] -- that creates an array key for the value field 21 (the value for that array key is 0 or the empty string), then it increments the field 21 value. The ++ has no effect on the array at all and can be removed. You probably intended a[$21]++, but it's not necessary that the array has a value for that key.– glenn jackman
Nov 30 at 19:36
Having said that, you do not actually ask a question in your question. Please read How do I ask a good question?
– glenn jackman
Nov 30 at 19:38
Having said that, you do not actually ask a question in your question. Please read How do I ask a good question?
– glenn jackman
Nov 30 at 19:38
@glennjackman Thanks for your help. I had made some typos. What I am trying to do is have both files compared and as I said in the post, File 1 has all of my data and File 2 $1 has the small group of data that I need to pull out of File 1 $7. When I try this and have File 2 with only 1 data point, it works, But when I have all 20K data points, I am not given any data from File1. I'll edit the main post with more information.
– Leo P
Nov 30 at 21:06
@glennjackman Thanks for your help. I had made some typos. What I am trying to do is have both files compared and as I said in the post, File 1 has all of my data and File 2 $1 has the small group of data that I need to pull out of File 1 $7. When I try this and have File 2 with only 1 data point, it works, But when I have all 20K data points, I am not given any data from File1. I'll edit the main post with more information.
– Leo P
Nov 30 at 21:06
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
Kindly use below command to get the required output.
awk -F "," 'NR==FNR {a[$7];next}($1 in a){print $1}' file1 file2 > newfile
edited Dec 1 at 11:54
Jeff Schaller
37k1052121
answered Dec 1 at 7:03
Praveen Kumar BS
1,166138
It seems that it does not really work the way i needed it to. The issue is that it still does not work when File2 has more than 1 entry. The only way it works is if File 2 has only 1 ID in the txt file. Another issue is that this way only prints out the line of File2 whereas I want to print the matching line in File1 that has the same ID as in File2. Thanks
– Leo P
yesterday
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
Kindly use below command to get the required output.
awk -F "," 'NR==FNR {a[$7];next}($1 in a){print $1}' file1 file2 > newfile
edited Dec 1 at 11:54
Jeff Schaller
37k1052121
answered Dec 1 at 7:03
Praveen Kumar BS
1,166138
It seems that it does not really work the way i needed it to. The issue is that it still does not work when File2 has more than 1 entry. The only way it works is if File 2 has only 1 ID in the txt file. Another issue is that this way only prints out the line of File2 whereas I want to print the matching line in File1 that has the same ID as in File2. Thanks
– Leo P
yesterday
add a comment |
up vote
0
down vote
Kindly use below command to get the required output.
awk -F "," 'NR==FNR {a[$7];next}($1 in a){print $1}' file1 file2 > newfile
edited Dec 1 at 11:54
Jeff Schaller
37k1052121
answered Dec 1 at 7:03
Praveen Kumar BS
1,166138
It seems that it does not really work the way i needed it to. The issue is that it still does not work when File2 has more than 1 entry. The only way it works is if File 2 has only 1 ID in the txt file. Another issue is that this way only prints out the line of File2 whereas I want to print the matching line in File1 that has the same ID as in File2. Thanks
– Leo P
yesterday
add a comment |
up vote
0
down vote
up vote
0
down vote
Kindly use below command to get the required output.
awk -F "," 'NR==FNR {a[$7];next}($1 in a){print $1}' file1 file2 > newfile
edited Dec 1 at 11:54
Jeff Schaller
37k1052121
answered Dec 1 at 7:03
Praveen Kumar BS
1,166138
Kindly use below command to get the required output.
awk -F "," 'NR==FNR {a[$7];next}($1 in a){print $1}' file1 file2 > newfile
edited Dec 1 at 11:54
Jeff Schaller
37k1052121
answered Dec 1 at 7:03
Praveen Kumar BS
1,166138
edited Dec 1 at 11:54
Jeff Schaller
37k1052121
edited Dec 1 at 11:54
Jeff Schaller
37k1052121
edited Dec 1 at 11:54
Jeff Schaller
37k1052121
37k1052121
answered Dec 1 at 7:03
Praveen Kumar BS
1,166138
answered Dec 1 at 7:03
Praveen Kumar BS
1,166138
answered Dec 1 at 7:03
Praveen Kumar BS
1,166138
1,166138
It seems that it does not really work the way i needed it to. The issue is that it still does not work when File2 has more than 1 entry. The only way it works is if File 2 has only 1 ID in the txt file. Another issue is that this way only prints out the line of File2 whereas I want to print the matching line in File1 that has the same ID as in File2. Thanks
– Leo P
yesterday
add a comment |
It seems that it does not really work the way i needed it to. The issue is that it still does not work when File2 has more than 1 entry. The only way it works is if File 2 has only 1 ID in the txt file. Another issue is that this way only prints out the line of File2 whereas I want to print the matching line in File1 that has the same ID as in File2. Thanks
– Leo P
yesterday
It seems that it does not really work the way i needed it to. The issue is that it still does not work when File2 has more than 1 entry. The only way it works is if File 2 has only 1 ID in the txt file. Another issue is that this way only prints out the line of File2 whereas I want to print the matching line in File1 that has the same ID as in File2. Thanks
– Leo P
yesterday
It seems that it does not really work the way i needed it to. The issue is that it still does not work when File2 has more than 1 entry. The only way it works is if File 2 has only 1 ID in the txt file. Another issue is that this way only prints out the line of File2 whereas I want to print the matching line in File1 that has the same ID as in File2. Thanks
– Leo P
yesterday
add a comment |
Leo P is a new contributor. Be nice, and check out our Code of Conduct.
draft saved
draft discarded
Leo P is a new contributor. Be nice, and check out our Code of Conduct.
Leo P is a new contributor. Be nice, and check out our Code of Conduct.
Leo P is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f485228%2fcompare-2-files-and-print-matching-values-to-a-new-file%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Are you aware that
'Fshould be-F? Is that just a typo?– glenn jackman
Nov 30 at 19:34
This is not a bug, but it's a bit sloppy:
a[$21++]-- that creates an array key for the value field 21 (the value for that array key is 0 or the empty string), then it increments the field 21 value. The++has no effect on the array at all and can be removed. You probably intendeda[$21]++, but it's not necessary that the array has a value for that key.– glenn jackman
Nov 30 at 19:36
Having said that, you do not actually ask a question in your question. Please read How do I ask a good question?
– glenn jackman
Nov 30 at 19:38
@glennjackman Thanks for your help. I had made some typos. What I am trying to do is have both files compared and as I said in the post, File 1 has all of my data and File 2 $1 has the small group of data that I need to pull out of File 1 $7. When I try this and have File 2 with only 1 data point, it works, But when I have all 20K data points, I am not given any data from File1. I'll edit the main post with more information.
– Leo P
Nov 30 at 21:06