Run a program and log to both screen and file in real time











up vote
2
down vote

favorite












I've written a very simple python program (in reality my program is a lot more complex):





from time import sleep
for i in range(10):
print(i)
sleep(1)


to print the digits 0-9, 1 each second over 10 seconds. I've saved it as TestShell.py



I can run it in powershell using



python .TestShell.py


which outputs correctly, 1 digit each second. However I'd also like to store the output in a log file. I expected to use the Tee-Object for this



python .TestShell.py | Tee-Object -FilePath .fileLog.txt


which takes the log file as one parameter, and implicitly writes to console without a 2nd parameter. However now, no output is printed until the entire program is run. That is, I see nothing for 10 seconds, then the numbers 0-9 all at once.



Is there any way to run a (python) file, outputting to the terminal and a log file in real time?










share|improve this question
























  • Honestly, I would add the logging in the script itself.
    – LPChip
    2 days ago






  • 1




    @LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly what tee would do - maybe some different parameters?
    – Greedo
    2 days ago

















up vote
2
down vote

favorite












I've written a very simple python program (in reality my program is a lot more complex):





from time import sleep
for i in range(10):
print(i)
sleep(1)


to print the digits 0-9, 1 each second over 10 seconds. I've saved it as TestShell.py



I can run it in powershell using



python .TestShell.py


which outputs correctly, 1 digit each second. However I'd also like to store the output in a log file. I expected to use the Tee-Object for this



python .TestShell.py | Tee-Object -FilePath .fileLog.txt


which takes the log file as one parameter, and implicitly writes to console without a 2nd parameter. However now, no output is printed until the entire program is run. That is, I see nothing for 10 seconds, then the numbers 0-9 all at once.



Is there any way to run a (python) file, outputting to the terminal and a log file in real time?










share|improve this question
























  • Honestly, I would add the logging in the script itself.
    – LPChip
    2 days ago






  • 1




    @LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly what tee would do - maybe some different parameters?
    – Greedo
    2 days ago















up vote
2
down vote

favorite









up vote
2
down vote

favorite











I've written a very simple python program (in reality my program is a lot more complex):





from time import sleep
for i in range(10):
print(i)
sleep(1)


to print the digits 0-9, 1 each second over 10 seconds. I've saved it as TestShell.py



I can run it in powershell using



python .TestShell.py


which outputs correctly, 1 digit each second. However I'd also like to store the output in a log file. I expected to use the Tee-Object for this



python .TestShell.py | Tee-Object -FilePath .fileLog.txt


which takes the log file as one parameter, and implicitly writes to console without a 2nd parameter. However now, no output is printed until the entire program is run. That is, I see nothing for 10 seconds, then the numbers 0-9 all at once.



Is there any way to run a (python) file, outputting to the terminal and a log file in real time?










share|improve this question















I've written a very simple python program (in reality my program is a lot more complex):





from time import sleep
for i in range(10):
print(i)
sleep(1)


to print the digits 0-9, 1 each second over 10 seconds. I've saved it as TestShell.py



I can run it in powershell using



python .TestShell.py


which outputs correctly, 1 digit each second. However I'd also like to store the output in a log file. I expected to use the Tee-Object for this



python .TestShell.py | Tee-Object -FilePath .fileLog.txt


which takes the log file as one parameter, and implicitly writes to console without a 2nd parameter. However now, no output is printed until the entire program is run. That is, I see nothing for 10 seconds, then the numbers 0-9 all at once.



Is there any way to run a (python) file, outputting to the terminal and a log file in real time?







windows-10 powershell python






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 2 days ago

























asked 2 days ago









Greedo

235315




235315












  • Honestly, I would add the logging in the script itself.
    – LPChip
    2 days ago






  • 1




    @LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly what tee would do - maybe some different parameters?
    – Greedo
    2 days ago




















  • Honestly, I would add the logging in the script itself.
    – LPChip
    2 days ago






  • 1




    @LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly what tee would do - maybe some different parameters?
    – Greedo
    2 days ago


















Honestly, I would add the logging in the script itself.
– LPChip
2 days ago




Honestly, I would add the logging in the script itself.
– LPChip
2 days ago




1




1




@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly what tee would do - maybe some different parameters?
– Greedo
2 days ago






@LPChip I thought about that, but in the real situation, most of the program output comes from a library I reference - I don't want to have to look inside it to change the logging. Even if I overload the default logger with a custom one, I have no way of guaranteeing the library isn't doing that too. So I really need a generic approach that monitors stdout and copies it to a file. I thought that was exactly what tee would do - maybe some different parameters?
– Greedo
2 days ago












2 Answers
2






active

oldest

votes

















up vote
1
down vote













The problem is that the PowerShell Tee-Object doesn't flush the output stream,
but only waits for the source to do that, so you get the output at the end of the
operation. This is by design.



Use this syntax instead to write the output line-by-line :



python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt





share|improve this answer























  • Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing " is a typo
    – Greedo
    2 days ago










  • Is this an answer that is in the context to the original poster’s self answer?
    – JakeGould
    2 days ago










  • Should be working. Strange.
    – harrymc
    2 days ago


















up vote
1
down vote













Well one approach can be found here



This is to use the -u switch to change the buffering mode of python itself, giving



python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt


But that isn't a solution for all languages






share|improve this answer























    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "3"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














     

    draft saved


    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1376241%2frun-a-program-and-log-to-both-screen-and-file-in-real-time%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    1
    down vote













    The problem is that the PowerShell Tee-Object doesn't flush the output stream,
    but only waits for the source to do that, so you get the output at the end of the
    operation. This is by design.



    Use this syntax instead to write the output line-by-line :



    python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt





    share|improve this answer























    • Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing " is a typo
      – Greedo
      2 days ago










    • Is this an answer that is in the context to the original poster’s self answer?
      – JakeGould
      2 days ago










    • Should be working. Strange.
      – harrymc
      2 days ago















    up vote
    1
    down vote













    The problem is that the PowerShell Tee-Object doesn't flush the output stream,
    but only waits for the source to do that, so you get the output at the end of the
    operation. This is by design.



    Use this syntax instead to write the output line-by-line :



    python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt





    share|improve this answer























    • Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing " is a typo
      – Greedo
      2 days ago










    • Is this an answer that is in the context to the original poster’s self answer?
      – JakeGould
      2 days ago










    • Should be working. Strange.
      – harrymc
      2 days ago













    up vote
    1
    down vote










    up vote
    1
    down vote









    The problem is that the PowerShell Tee-Object doesn't flush the output stream,
    but only waits for the source to do that, so you get the output at the end of the
    operation. This is by design.



    Use this syntax instead to write the output line-by-line :



    python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt





    share|improve this answer














    The problem is that the PowerShell Tee-Object doesn't flush the output stream,
    but only waits for the source to do that, so you get the output at the end of the
    operation. This is by design.



    Use this syntax instead to write the output line-by-line :



    python .TestShell.py | ForEach-Object { Write-Host $_; $_} | Set-Content .fileLog.txt






    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited 2 days ago

























    answered 2 days ago









    harrymc

    247k10256542




    247k10256542












    • Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing " is a typo
      – Greedo
      2 days ago










    • Is this an answer that is in the context to the original poster’s self answer?
      – JakeGould
      2 days ago










    • Should be working. Strange.
      – harrymc
      2 days ago


















    • Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing " is a typo
      – Greedo
      2 days ago










    • Is this an answer that is in the context to the original poster’s self answer?
      – JakeGould
      2 days ago










    • Should be working. Strange.
      – harrymc
      2 days ago
















    Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing " is a typo
    – Greedo
    2 days ago




    Doesn't seem to be working - both the log and the terminal are only written to at the end of the execution :/ PS I assume that trailing " is a typo
    – Greedo
    2 days ago












    Is this an answer that is in the context to the original poster’s self answer?
    – JakeGould
    2 days ago




    Is this an answer that is in the context to the original poster’s self answer?
    – JakeGould
    2 days ago












    Should be working. Strange.
    – harrymc
    2 days ago




    Should be working. Strange.
    – harrymc
    2 days ago












    up vote
    1
    down vote













    Well one approach can be found here



    This is to use the -u switch to change the buffering mode of python itself, giving



    python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt


    But that isn't a solution for all languages






    share|improve this answer



























      up vote
      1
      down vote













      Well one approach can be found here



      This is to use the -u switch to change the buffering mode of python itself, giving



      python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt


      But that isn't a solution for all languages






      share|improve this answer

























        up vote
        1
        down vote










        up vote
        1
        down vote









        Well one approach can be found here



        This is to use the -u switch to change the buffering mode of python itself, giving



        python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt


        But that isn't a solution for all languages






        share|improve this answer














        Well one approach can be found here



        This is to use the -u switch to change the buffering mode of python itself, giving



        python -u .TestShell.py | Tee-Object -FilePath .fileLog.txt


        But that isn't a solution for all languages







        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited 2 days ago

























        answered 2 days ago









        Greedo

        235315




        235315






























             

            draft saved


            draft discarded



















































             


            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1376241%2frun-a-program-and-log-to-both-screen-and-file-in-real-time%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Accessing regular linux commands in Huawei's Dopra Linux

            Can't connect RFCOMM socket: Host is down

            Kernel panic - not syncing: Fatal Exception in Interrupt