Topic: Feature Request: StdErr & Stdout when using ShellExecute  (Read 278 times)

0 Members and 1 Guest are viewing this topic.

Feature Request: StdErr & Stdout when using ShellExecute
« on: December 13, 2017, 03:28:26 PM »

Homes32

  • Code Baker
  • Chef
  • ***
  • Date Registered: Dec 2010
  • Posts: 147
On my wish-list since "forever" has been a way to redirect stdout/stderr into the builder, or at least log the output without using tricks like running from .cmd/.bat using redirection or using an external app to pipe them off and tee the output to another file for programs that don't write their own logs.

I hate with the fury of a thousand suns when applications/console windows/etc popup during a build, steal focus, annoy me while I'm trying to work on something else, etc. My brain likes to be busy, so I usually run the builder in the background and work on something else in the meantime. However some applications do display useful information while running (makeiso is a good example), so developers feel the need to run them "unhidden".

I see several possible options:

1. Always log stdout/stderr to the log
2. Publish the current contents of stderr/stdout to a small text area on the progressUI possibly under the messages section. This would be useful for programs like makeiso, 7z, etc that show progress that may be useful or entertaining. Also output to log. [See attached screen mockup for a crude example.]
3. Global builder setting to toggle logging of stdout/stderr
4. Add an optional argument to shellexecute telling PeBakery what to do with stdout/stderr
5. Return the streams in a variable for the developer to deal with. (I don't see much value in this. double yuck if the program has a large output this could cause unwanted slowdowns/hangs)

I prefer option #2  :smile:

This could possible cause a small amount of overhead reading the stream and flushing it to the log (as with any logging action), but I would much rather have everything in one spot if possible.
real-time reading of the streams would be preferred to just collecting them in a buffer and dumping them the log at the end of the process, to avoid having the builder to hang if the called program has a large amount of output.

Discussion is welcome. If something solid can be agreed on I will submit a ticket to the tracker for future development.
« Last Edit: December 13, 2017, 03:40:53 PM by Homes32 »

Re: Feature Request: StdErr & Stdout when using ShellExecute
« Reply #1 on: December 14, 2017, 01:21:36 AM »

Lancelot

  • Moderator, Gena Baker
  • Grand Chef
  • *****
  • Date Registered: Sep 2010
  • Posts: 7756
I feel It will be nice to get cmd applications have Echo on log  :thumbsup:
Maybe as default PEBakery can detect cmd application and follow output with something like Echo lines.
This will give us more ideas .....

ps: The only reason I did not implement to Macro Library with tee.exe was to avoid another False Positive AntiVirus or Firewall .....

*
I hate with the fury of a thousand suns when applications/console windows/etc popup during a build, steal focus, annoy me while I'm trying to work on something else, etc.

I had same feeling for years, and fixed many relevant plugins with workaround.
Let me know which application/console popup now on which plugins. ?

:turtle:

Re: Feature Request: StdErr & Stdout when using ShellExecute
« Reply #2 on: December 14, 2017, 02:34:47 AM »

Galapo

  • Gena Baker
  • Grand Chef
  • *****
  • Location: Australia
  • Date Registered: Sep 2010
  • Posts: 2014
Yes, this is a good idea to me as well.

I guess #2, #3, and #4 could all be implemented together.

Re: Feature Request: StdErr & Stdout when using ShellExecute
« Reply #3 on: December 14, 2017, 11:57:46 AM »

Homes32

  • Code Baker
  • Chef
  • ***
  • Date Registered: Dec 2010
  • Posts: 147
I feel It will be nice to get cmd applications have Echo on log  :thumbsup:
Maybe as default PEBakery can detect cmd application and follow output with something like Echo lines.
This will give us more ideas .....
This is already possible with c#
something like this
Code: [Select]
proc.StartInfo.UseShellExecute = false;
proc.StartInfo.Verb = "Open";
proc.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
proc.StartInfo.CreateNoWindow = true;

//  redirect stdout/err to buffer
proc.StartInfo.RedirectStandardError = true;
proc.StartInfo.RedirectStandardOutput = true;


proc.Start()
do
{
   // read the streams and do something with them
   proc.StandardOutput.ReadLine()
   proc.StandardError.ReadToEnd()
}
while (!$proc.HasExited)
so not a question of "can we do it?" more of a "what is the most useful way to present the information for view/logging with minimal to no impact on performance?"

ps: The only reason I did not implement to Macro Library with tee.exe was to avoid another False Positive AntiVirus or Firewall .....
you hit another reasoning right on the head!  :thumbsup:
Anytime you add an homemade or external program to the mix you have the very likely possibility for sloppy/lazy/stupid antivirus vendors to flag your exe as a false positive.
« Last Edit: December 14, 2017, 11:58:51 AM by Homes32 »

Re: Feature Request: StdErr & Stdout when using ShellExecute
« Reply #4 on: December 14, 2017, 05:15:44 PM »

Lancelot

  • Moderator, Gena Baker
  • Grand Chef
  • *****
  • Date Registered: Sep 2010
  • Posts: 7756
Anytime you add an homemade or external program to the mix you have the very likely possibility for sloppy/lazy/stupid antivirus vendors to flag your exe as a false positive.
We only decrease possibilities not all.
So we can have a short list of false positives.
ex: c't could make a short list for av false positives on their cd distribution
--> That is the reason I did not add tee or other things to Macro Library.

ps: We had unupxed all executables in very past to avoid some of false positives.

not all ex: Recently I found 7zip + autoit 3361 and winbuilder cause false positive a firewall ......

Shortly:
PEBakery will not end false positives.
It is only design of projects and plugins that lower the amount of false positives.



*
Going back to subject:
What I mean is this:

ila_rendered

This will:
Get percentage etc. from cmd title (which % not always available ex: Win10PESE default CreateISO ... )
Get screen info inside PEBakery window
Most important:
screen info also available at log (= tee + echo file => to get )

I do not think there will be significant speed loss during build for cmd applications (+.cmd)

*
To me this request is not urgent still very very valuable.



*
*
I hate with the fury of a thousand suns when applications/console windows/etc popup during a build, steal focus, annoy me while I'm trying to work on something else, etc.

I had same feeling for years, and fixed many relevant plugins with workaround.
Let me know which application/console popup now on which plugins. ?

"fury of a thousand suns when applications/console windows/etc popup during a build"
->
Let me know which application/console popup now on which plugins. ?

:turtle:
« Last Edit: December 14, 2017, 05:16:46 PM by Lancelot »

Re: Feature Request: StdErr & Stdout when using ShellExecute
« Reply #5 on: December 15, 2017, 11:38:19 AM »

ied206

  • Moderator,
  • Chef
  • *****
  • Location: South Korea
  • Date Registered: Jan 2014
  • Posts: 203
Experimental support for redirect of stdout in ShellExecute,Hide is added in commit cffb428.

Currently only stdout is shown in interface.

Screenshot
ila_rendered

stdout and stderr will be logged.
Code: [Select]
ShellExecute,Hide,ping,"8.8.8.8 -n 16"
ShellExecute,Hide,cmd,"/c dir //"

Code: [Select]
<Code Logs>
[<Code Logs>
[1/1] Processing Plugin [CodeBox TheOven.org] (Win10PESE\CodeBox.txt)
--------------------------------------------------------------------------------
[Info] Processing Section [Process]
  [Success] If - File [E:\WinPE_dev\Win10PESE\Projects\Win10PESE\Build\Macro_Library.script] exists (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Global variable [%Shc_Mode%] set to [0] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Info] AddVariables - Import Macro from [ApiVar] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [_Calculate] set to [Run,%API%,Calculate] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [Add_Shortcut] set to [Run,%API%,AddShortcut%_ML_Shortcut_TYPE%] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [Add_Pin] set to [Run,%Api%,Add_Pin_Process%API_TYPE%] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [Add] set to [Run,%API%,Add_] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [AddAutoRun] set to [Run,%API%,Add_Auto_Run%API_TYPE%] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [AddPostConfig] set to [Run,%API%,Add_Post_Config%API_TYPE%] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [Arch] set to [Run,%API%,Arch_check] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [Call] set to [Run,%API%,_CenterCa] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [RegAddBoot] set to [Run,%API%,RegAdd_boot_%_ML_RegAddBootMethod%] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [Require_FileQ] set to [Run,%API%,Expand_FileQ%API_TYPE%%SourceArch%%TargetWOW64%%PluginArch%] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [Require] set to [Run,%API%,Expand_FileX%API_TYPE%] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [RunFrom] set to [Run,%API%,RunFrom_%_ML_RF_TYPE%] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [ReadENV] set to [Run,%API%,Read_ENV%API_TYPE%] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Success] AddVariables - Local macro [ChangeInterface] set to [Run,%API%,_ScriptInterface] (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    [Info] AddVariables - Imported 14 Macro (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
    AddVariables - -------------------------------------------------------------------------------- (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
  [Info] If - End of CodeBlock (If,ExistFile,%api%,AddVariables,%api%,ApiVar,GLOBAL)
  [Ignore] Comment (//--)
  [Success] ShellExecute - Executed [ping 8.8.8.8 -n 16], returned exit code [0], took [0s] (ShellExecute,Hide,ping,"8.8.8.8 -n 16")
  [Success] ShellExecute - Exit code [0] saved into variable [%ExitCode%] (ShellExecute,Hide,ping,"8.8.8.8 -n 16")
  [Success] ShellExecute - [Standard Output]
Ping 8.8.8.8 32바이트 데이터 사용:
8.8.8.8의 응답: 바이트=32 시간=33ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=34ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=34ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=33ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=33ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=33ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=34ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=34ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=33ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=33ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=33ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=33ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=33ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=34ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=33ms TTL=57
8.8.8.8의 응답: 바이트=32 시간=34ms TTL=57

8.8.8.8에 대한 Ping 통계:
    패킷: 보냄 = 16, 받음 = 16, 손실 = 0 (0% 손실),
왕복 시간(밀리초):
    최소 = 33ms, 최대 = 34ms, 평균 = 33ms
 (ShellExecute,Hide,ping,"8.8.8.8 -n 16")
  [Success] ShellExecute - Executed [cmd /c dir //], returned exit code [1], took [0s] (ShellExecute,Hide,cmd,"/c dir //")
  [Success] ShellExecute - Exit code [1] saved into variable [%ExitCode%] (ShellExecute,Hide,cmd,"/c dir //")
  [Success] ShellExecute - [Standard Error] 스위치가 틀립니다 - "/". (ShellExecute,Hide,cmd,"/c dir //")
[Info] End of Section [Process]
End of Plugin [Win10PESE\CodeBox.txt]
--------------------------------------------------------------------------------
- Local Variables of Plugin [CodeBox TheOven.org]
%ExitCode% = 1
--------------------------------------------------------------------------------


 

Powered by EzPortal