How to delete a line with variable search using sed command
up vote
2
down vote
favorite
What is wrong with the sed
command on deleting the line which matches the input data?
InputData.txt
123,
1234,
1453,
Datatodelete.txt
1234,hellofirstline
123,hellosecondline
14676,hellothirdline
1453,hellofourthline
expected output in the Datatodelete.txt
14676,hellothirdline
Script:
echo "the script starts now"
while read EachLine
do
echo $EachLine
sed "/$EachLine/d" < /home/Datatodelete.txt >/home/dummy
done < /home/InputData.txt
shell-script sed
add a comment |
up vote
2
down vote
favorite
What is wrong with the sed
command on deleting the line which matches the input data?
InputData.txt
123,
1234,
1453,
Datatodelete.txt
1234,hellofirstline
123,hellosecondline
14676,hellothirdline
1453,hellofourthline
expected output in the Datatodelete.txt
14676,hellothirdline
Script:
echo "the script starts now"
while read EachLine
do
echo $EachLine
sed "/$EachLine/d" < /home/Datatodelete.txt >/home/dummy
done < /home/InputData.txt
shell-script sed
add a comment |
up vote
2
down vote
favorite
up vote
2
down vote
favorite
What is wrong with the sed
command on deleting the line which matches the input data?
InputData.txt
123,
1234,
1453,
Datatodelete.txt
1234,hellofirstline
123,hellosecondline
14676,hellothirdline
1453,hellofourthline
expected output in the Datatodelete.txt
14676,hellothirdline
Script:
echo "the script starts now"
while read EachLine
do
echo $EachLine
sed "/$EachLine/d" < /home/Datatodelete.txt >/home/dummy
done < /home/InputData.txt
shell-script sed
What is wrong with the sed
command on deleting the line which matches the input data?
InputData.txt
123,
1234,
1453,
Datatodelete.txt
1234,hellofirstline
123,hellosecondline
14676,hellothirdline
1453,hellofourthline
expected output in the Datatodelete.txt
14676,hellothirdline
Script:
echo "the script starts now"
while read EachLine
do
echo $EachLine
sed "/$EachLine/d" < /home/Datatodelete.txt >/home/dummy
done < /home/InputData.txt
shell-script sed
shell-script sed
edited Nov 25 at 14:46
Rui F Ribeiro
38.3k1475126
38.3k1475126
asked Mar 19 '14 at 15:06
user3438085
11112
11112
add a comment |
add a comment |
3 Answers
3
active
oldest
votes
up vote
5
down vote
Your sed
command doesn't work because during the loop, each time it reads a line, it deletes that line (and only that line) from the full input file, outputting it to /home/dummy
. This means that the output file gets overwritten each time. So the first iteration of the loop removes the line starting with 123, but then the second iteration uses the original full file which still includes this line.
Try grep
instead:
grep -vFf /home/InputData.txt /home/Datatodelete.txt > /home/dummy
From man grep
:
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by
newlines, any of which is to be matched. (-F is specified by
POSIX.)
-f FILE, --file=FILE
Obtain patterns from FILE, one per line. The empty file
contains zero patterns, and therefore matches nothing. (-f is
specified by POSIX.)
-v, --invert-match
Invert the sense of matching, to select non-matching lines. (-v
is specified by POSIX.)
2
Depending on what else is inInput.txt
you may want to usegrep -F
if there are characters that are intended to be matched verbatim. Eg.
– Graeme
Mar 19 '14 at 15:17
The above grep command is still not removing the records from Datatodelete.txt. It is still printing the whole content in dummy file
– user3438085
Mar 19 '14 at 21:00
add a comment |
up vote
2
down vote
An awk
version:
awk -F, 'FNR==NR{a[$1];next} !($1 in a)' InputData.txt Datatodelete.txt
This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post.
– Flup
Mar 19 '14 at 15:45
@Flup, it does provide the output the OP seeks.
– glenn jackman
Mar 19 '14 at 15:46
...but doesn't answer the OP's question.
– Flup
Mar 19 '14 at 15:47
1
@Flup This isn't a story-telling competition.
– devnull
Mar 19 '14 at 16:22
add a comment |
up vote
1
down vote
First, you don't want while read
in there anywhere - sed
will read your file. Next, you need to make sure that you handle greedy
matches - sed
will pull in as much as it can. So
echo 1234 | sed -n '/123/p'
1234
See? It prints it.
So you need, based on what you've shown, something like this:
</home/InputData.txt
sed -n '/1234/s//& hellofirstline/p;
/123[^4]/s//& hellosecondline/p;
/14676/s//& hellothirdline/p;
/1453/s//& hellofourthline/p' >/home/dummy
If your sed
script is in a file:
</home/InputData.txt
sed -nf ./delete.sed >/home/dummy
add a comment |
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
5
down vote
Your sed
command doesn't work because during the loop, each time it reads a line, it deletes that line (and only that line) from the full input file, outputting it to /home/dummy
. This means that the output file gets overwritten each time. So the first iteration of the loop removes the line starting with 123, but then the second iteration uses the original full file which still includes this line.
Try grep
instead:
grep -vFf /home/InputData.txt /home/Datatodelete.txt > /home/dummy
From man grep
:
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by
newlines, any of which is to be matched. (-F is specified by
POSIX.)
-f FILE, --file=FILE
Obtain patterns from FILE, one per line. The empty file
contains zero patterns, and therefore matches nothing. (-f is
specified by POSIX.)
-v, --invert-match
Invert the sense of matching, to select non-matching lines. (-v
is specified by POSIX.)
2
Depending on what else is inInput.txt
you may want to usegrep -F
if there are characters that are intended to be matched verbatim. Eg.
– Graeme
Mar 19 '14 at 15:17
The above grep command is still not removing the records from Datatodelete.txt. It is still printing the whole content in dummy file
– user3438085
Mar 19 '14 at 21:00
add a comment |
up vote
5
down vote
Your sed
command doesn't work because during the loop, each time it reads a line, it deletes that line (and only that line) from the full input file, outputting it to /home/dummy
. This means that the output file gets overwritten each time. So the first iteration of the loop removes the line starting with 123, but then the second iteration uses the original full file which still includes this line.
Try grep
instead:
grep -vFf /home/InputData.txt /home/Datatodelete.txt > /home/dummy
From man grep
:
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by
newlines, any of which is to be matched. (-F is specified by
POSIX.)
-f FILE, --file=FILE
Obtain patterns from FILE, one per line. The empty file
contains zero patterns, and therefore matches nothing. (-f is
specified by POSIX.)
-v, --invert-match
Invert the sense of matching, to select non-matching lines. (-v
is specified by POSIX.)
2
Depending on what else is inInput.txt
you may want to usegrep -F
if there are characters that are intended to be matched verbatim. Eg.
– Graeme
Mar 19 '14 at 15:17
The above grep command is still not removing the records from Datatodelete.txt. It is still printing the whole content in dummy file
– user3438085
Mar 19 '14 at 21:00
add a comment |
up vote
5
down vote
up vote
5
down vote
Your sed
command doesn't work because during the loop, each time it reads a line, it deletes that line (and only that line) from the full input file, outputting it to /home/dummy
. This means that the output file gets overwritten each time. So the first iteration of the loop removes the line starting with 123, but then the second iteration uses the original full file which still includes this line.
Try grep
instead:
grep -vFf /home/InputData.txt /home/Datatodelete.txt > /home/dummy
From man grep
:
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by
newlines, any of which is to be matched. (-F is specified by
POSIX.)
-f FILE, --file=FILE
Obtain patterns from FILE, one per line. The empty file
contains zero patterns, and therefore matches nothing. (-f is
specified by POSIX.)
-v, --invert-match
Invert the sense of matching, to select non-matching lines. (-v
is specified by POSIX.)
Your sed
command doesn't work because during the loop, each time it reads a line, it deletes that line (and only that line) from the full input file, outputting it to /home/dummy
. This means that the output file gets overwritten each time. So the first iteration of the loop removes the line starting with 123, but then the second iteration uses the original full file which still includes this line.
Try grep
instead:
grep -vFf /home/InputData.txt /home/Datatodelete.txt > /home/dummy
From man grep
:
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by
newlines, any of which is to be matched. (-F is specified by
POSIX.)
-f FILE, --file=FILE
Obtain patterns from FILE, one per line. The empty file
contains zero patterns, and therefore matches nothing. (-f is
specified by POSIX.)
-v, --invert-match
Invert the sense of matching, to select non-matching lines. (-v
is specified by POSIX.)
edited Mar 19 '14 at 15:33
terdon♦
126k31243419
126k31243419
answered Mar 19 '14 at 15:12
Josh Jolly
1,312812
1,312812
2
Depending on what else is inInput.txt
you may want to usegrep -F
if there are characters that are intended to be matched verbatim. Eg.
– Graeme
Mar 19 '14 at 15:17
The above grep command is still not removing the records from Datatodelete.txt. It is still printing the whole content in dummy file
– user3438085
Mar 19 '14 at 21:00
add a comment |
2
Depending on what else is inInput.txt
you may want to usegrep -F
if there are characters that are intended to be matched verbatim. Eg.
– Graeme
Mar 19 '14 at 15:17
The above grep command is still not removing the records from Datatodelete.txt. It is still printing the whole content in dummy file
– user3438085
Mar 19 '14 at 21:00
2
2
Depending on what else is in
Input.txt
you may want to use grep -F
if there are characters that are intended to be matched verbatim. Eg .
– Graeme
Mar 19 '14 at 15:17
Depending on what else is in
Input.txt
you may want to use grep -F
if there are characters that are intended to be matched verbatim. Eg .
– Graeme
Mar 19 '14 at 15:17
The above grep command is still not removing the records from Datatodelete.txt. It is still printing the whole content in dummy file
– user3438085
Mar 19 '14 at 21:00
The above grep command is still not removing the records from Datatodelete.txt. It is still printing the whole content in dummy file
– user3438085
Mar 19 '14 at 21:00
add a comment |
up vote
2
down vote
An awk
version:
awk -F, 'FNR==NR{a[$1];next} !($1 in a)' InputData.txt Datatodelete.txt
This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post.
– Flup
Mar 19 '14 at 15:45
@Flup, it does provide the output the OP seeks.
– glenn jackman
Mar 19 '14 at 15:46
...but doesn't answer the OP's question.
– Flup
Mar 19 '14 at 15:47
1
@Flup This isn't a story-telling competition.
– devnull
Mar 19 '14 at 16:22
add a comment |
up vote
2
down vote
An awk
version:
awk -F, 'FNR==NR{a[$1];next} !($1 in a)' InputData.txt Datatodelete.txt
This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post.
– Flup
Mar 19 '14 at 15:45
@Flup, it does provide the output the OP seeks.
– glenn jackman
Mar 19 '14 at 15:46
...but doesn't answer the OP's question.
– Flup
Mar 19 '14 at 15:47
1
@Flup This isn't a story-telling competition.
– devnull
Mar 19 '14 at 16:22
add a comment |
up vote
2
down vote
up vote
2
down vote
An awk
version:
awk -F, 'FNR==NR{a[$1];next} !($1 in a)' InputData.txt Datatodelete.txt
An awk
version:
awk -F, 'FNR==NR{a[$1];next} !($1 in a)' InputData.txt Datatodelete.txt
answered Mar 19 '14 at 15:20
cuonglm
101k23196297
101k23196297
This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post.
– Flup
Mar 19 '14 at 15:45
@Flup, it does provide the output the OP seeks.
– glenn jackman
Mar 19 '14 at 15:46
...but doesn't answer the OP's question.
– Flup
Mar 19 '14 at 15:47
1
@Flup This isn't a story-telling competition.
– devnull
Mar 19 '14 at 16:22
add a comment |
This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post.
– Flup
Mar 19 '14 at 15:45
@Flup, it does provide the output the OP seeks.
– glenn jackman
Mar 19 '14 at 15:46
...but doesn't answer the OP's question.
– Flup
Mar 19 '14 at 15:47
1
@Flup This isn't a story-telling competition.
– devnull
Mar 19 '14 at 16:22
This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post.
– Flup
Mar 19 '14 at 15:45
This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post.
– Flup
Mar 19 '14 at 15:45
@Flup, it does provide the output the OP seeks.
– glenn jackman
Mar 19 '14 at 15:46
@Flup, it does provide the output the OP seeks.
– glenn jackman
Mar 19 '14 at 15:46
...but doesn't answer the OP's question.
– Flup
Mar 19 '14 at 15:47
...but doesn't answer the OP's question.
– Flup
Mar 19 '14 at 15:47
1
1
@Flup This isn't a story-telling competition.
– devnull
Mar 19 '14 at 16:22
@Flup This isn't a story-telling competition.
– devnull
Mar 19 '14 at 16:22
add a comment |
up vote
1
down vote
First, you don't want while read
in there anywhere - sed
will read your file. Next, you need to make sure that you handle greedy
matches - sed
will pull in as much as it can. So
echo 1234 | sed -n '/123/p'
1234
See? It prints it.
So you need, based on what you've shown, something like this:
</home/InputData.txt
sed -n '/1234/s//& hellofirstline/p;
/123[^4]/s//& hellosecondline/p;
/14676/s//& hellothirdline/p;
/1453/s//& hellofourthline/p' >/home/dummy
If your sed
script is in a file:
</home/InputData.txt
sed -nf ./delete.sed >/home/dummy
add a comment |
up vote
1
down vote
First, you don't want while read
in there anywhere - sed
will read your file. Next, you need to make sure that you handle greedy
matches - sed
will pull in as much as it can. So
echo 1234 | sed -n '/123/p'
1234
See? It prints it.
So you need, based on what you've shown, something like this:
</home/InputData.txt
sed -n '/1234/s//& hellofirstline/p;
/123[^4]/s//& hellosecondline/p;
/14676/s//& hellothirdline/p;
/1453/s//& hellofourthline/p' >/home/dummy
If your sed
script is in a file:
</home/InputData.txt
sed -nf ./delete.sed >/home/dummy
add a comment |
up vote
1
down vote
up vote
1
down vote
First, you don't want while read
in there anywhere - sed
will read your file. Next, you need to make sure that you handle greedy
matches - sed
will pull in as much as it can. So
echo 1234 | sed -n '/123/p'
1234
See? It prints it.
So you need, based on what you've shown, something like this:
</home/InputData.txt
sed -n '/1234/s//& hellofirstline/p;
/123[^4]/s//& hellosecondline/p;
/14676/s//& hellothirdline/p;
/1453/s//& hellofourthline/p' >/home/dummy
If your sed
script is in a file:
</home/InputData.txt
sed -nf ./delete.sed >/home/dummy
First, you don't want while read
in there anywhere - sed
will read your file. Next, you need to make sure that you handle greedy
matches - sed
will pull in as much as it can. So
echo 1234 | sed -n '/123/p'
1234
See? It prints it.
So you need, based on what you've shown, something like this:
</home/InputData.txt
sed -n '/1234/s//& hellofirstline/p;
/123[^4]/s//& hellosecondline/p;
/14676/s//& hellothirdline/p;
/1453/s//& hellofourthline/p' >/home/dummy
If your sed
script is in a file:
</home/InputData.txt
sed -nf ./delete.sed >/home/dummy
edited Mar 19 '14 at 15:20
answered Mar 19 '14 at 15:14
mikeserv
45.1k566152
45.1k566152
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f120417%2fhow-to-delete-a-line-with-variable-search-using-sed-command%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown