Run a program and log to both screen and file in real time
up vote
2
down vote
favorite
I've written a very simple python program (in reality my program is a lot more complex):
from time import sleep
for i in range(10):
print(i)
sleep(1)
to print the digits 0-9, 1 each second over 10 seconds. I've saved it as TestShell.py
I can run it in powershell using
python .TestShell.py
which outputs correctly, 1 digit each second. However I'd also like to store the output in a log file. I expected to use the Tee-Object
for this
python .TestShell.py | Tee-Object -FilePath .fileLog.txt
which takes the log file as one parameter, and implicitly writes to console without a 2nd parameter. However now, no output is printed until the entire program is run. That is, I see nothing for 10 seconds, then the numbers 0-9 all at once.
Is there any way to run a (python) file, outputting to the terminal and a log file in real time?
windows-10 powershell python
add a comment |
up vote
2
down vote
favorite
I've written a very simple python program (in reality my program is a lot more complex):
from time import sleep
for i in range(10):
print(i)
sleep(1)
to print the digits 0-9, 1 each second over 10 seconds. I've saved it as TestShell.py
I can run it in powershell using
python .TestShell.py
which outputs correctly, 1 digit each second. However I'd also like to store the output in a log file. I expected to use the Tee-Object
for this
python .TestShell.py | Tee-Object -FilePath .fileLog.txt
which takes the log file as one parameter, and implicitly writes to console without a 2nd parameter. However now, no output is printed until the entire program is run. That is, I see nothing for 10 seconds, then the numbers 0-9 all at once.
Is there any way to run a (python) file, outputting to the terminal and a log file in real time?
windows-10 powershell python
Honestly, I would add the logging in the script itself.
– LPChip
2 days ago
1
@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly whattee
would do - maybe some different parameters?
– Greedo
2 days ago
add a comment |
up vote
2
down vote
favorite
up vote
2
down vote
favorite
I've written a very simple python program (in reality my program is a lot more complex):
from time import sleep
for i in range(10):
print(i)
sleep(1)
to print the digits 0-9, 1 each second over 10 seconds. I've saved it as TestShell.py
I can run it in powershell using
python .TestShell.py
which outputs correctly, 1 digit each second. However I'd also like to store the output in a log file. I expected to use the Tee-Object
for this
python .TestShell.py | Tee-Object -FilePath .fileLog.txt
which takes the log file as one parameter, and implicitly writes to console without a 2nd parameter. However now, no output is printed until the entire program is run. That is, I see nothing for 10 seconds, then the numbers 0-9 all at once.
Is there any way to run a (python) file, outputting to the terminal and a log file in real time?
windows-10 powershell python
I've written a very simple python program (in reality my program is a lot more complex):
from time import sleep
for i in range(10):
print(i)
sleep(1)
to print the digits 0-9, 1 each second over 10 seconds. I've saved it as TestShell.py
I can run it in powershell using
python .TestShell.py
which outputs correctly, 1 digit each second. However I'd also like to store the output in a log file. I expected to use the Tee-Object
for this
python .TestShell.py | Tee-Object -FilePath .fileLog.txt
which takes the log file as one parameter, and implicitly writes to console without a 2nd parameter. However now, no output is printed until the entire program is run. That is, I see nothing for 10 seconds, then the numbers 0-9 all at once.
Is there any way to run a (python) file, outputting to the terminal and a log file in real time?
windows-10 powershell python
windows-10 powershell python
edited 2 days ago
asked 2 days ago
Greedo
235315
235315
Honestly, I would add the logging in the script itself.
– LPChip
2 days ago
1
@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly whattee
would do - maybe some different parameters?
– Greedo
2 days ago
add a comment |
Honestly, I would add the logging in the script itself.
– LPChip
2 days ago
1
@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly whattee
would do - maybe some different parameters?
– Greedo
2 days ago
Honestly, I would add the logging in the script itself.
– LPChip
2 days ago
Honestly, I would add the logging in the script itself.
– LPChip
2 days ago
1
1
@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly what
tee
would do - maybe some different parameters?– Greedo
2 days ago
@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly what
tee
would do - maybe some different parameters?– Greedo
2 days ago
add a comment |
2 Answers
2
active
oldest
votes
up vote
1
down vote
The problem is that the PowerShell Tee-Object
doesn't flush the output stream,
but only waits for the source to do that, so you get the output at the end of the
operation. This is by design.
Use this syntax instead to write the output line-by-line :
python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing"
is a typo
– Greedo
2 days ago
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
2 days ago
Should be working. Strange.
– harrymc
2 days ago
add a comment |
up vote
1
down vote
Well one approach can be found here
This is to use the -u switch to change the buffering mode of python itself, giving
python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt
But that isn't a solution for all languages
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
The problem is that the PowerShell Tee-Object
doesn't flush the output stream,
but only waits for the source to do that, so you get the output at the end of the
operation. This is by design.
Use this syntax instead to write the output line-by-line :
python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing"
is a typo
– Greedo
2 days ago
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
2 days ago
Should be working. Strange.
– harrymc
2 days ago
add a comment |
up vote
1
down vote
The problem is that the PowerShell Tee-Object
doesn't flush the output stream,
but only waits for the source to do that, so you get the output at the end of the
operation. This is by design.
Use this syntax instead to write the output line-by-line :
python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing"
is a typo
– Greedo
2 days ago
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
2 days ago
Should be working. Strange.
– harrymc
2 days ago
add a comment |
up vote
1
down vote
up vote
1
down vote
The problem is that the PowerShell Tee-Object
doesn't flush the output stream,
but only waits for the source to do that, so you get the output at the end of the
operation. This is by design.
Use this syntax instead to write the output line-by-line :
python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt
The problem is that the PowerShell Tee-Object
doesn't flush the output stream,
but only waits for the source to do that, so you get the output at the end of the
operation. This is by design.
Use this syntax instead to write the output line-by-line :
python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt
edited 2 days ago
answered 2 days ago
harrymc
247k10256542
247k10256542
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing"
is a typo
– Greedo
2 days ago
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
2 days ago
Should be working. Strange.
– harrymc
2 days ago
add a comment |
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing"
is a typo
– Greedo
2 days ago
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
2 days ago
Should be working. Strange.
– harrymc
2 days ago
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing
"
is a typo– Greedo
2 days ago
Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing
"
is a typo– Greedo
2 days ago
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
2 days ago
Is this an answer that is in the context to the original poster’s self answer?
– JakeGould
2 days ago
Should be working. Strange.
– harrymc
2 days ago
Should be working. Strange.
– harrymc
2 days ago
add a comment |
up vote
1
down vote
Well one approach can be found here
This is to use the -u switch to change the buffering mode of python itself, giving
python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt
But that isn't a solution for all languages
add a comment |
up vote
1
down vote
Well one approach can be found here
This is to use the -u switch to change the buffering mode of python itself, giving
python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt
But that isn't a solution for all languages
add a comment |
up vote
1
down vote
up vote
1
down vote
Well one approach can be found here
This is to use the -u switch to change the buffering mode of python itself, giving
python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt
But that isn't a solution for all languages
Well one approach can be found here
This is to use the -u switch to change the buffering mode of python itself, giving
python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt
But that isn't a solution for all languages
edited 2 days ago
answered 2 days ago
Greedo
235315
235315
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1376241%2frun-a-program-and-log-to-both-screen-and-file-in-real-time%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Honestly, I would add the logging in the script itself.
– LPChip
2 days ago
1
@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly what
tee
would do - maybe some different parameters?– Greedo
2 days ago