Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - ied206

Pages: [1] 2 3 ... 6
Thank you for all advices.
I had been busy these days, sorry for late reply.

Deciding future syntax cannot be done by few people, and I have no exact answer, just ideas.
Hopefully, we have times and opinions to talk about future syntaxes.

Here are some factors which I am concerned:

1. Case of using other language syntax

1-1) Logging / Debugging
WB082 currently depends on logging for debugging.
(Of course, it has limitations, obviously)
If we borrow other language, how can we integrate debugging functionaility to PEBakery?
If newcomers (who are not familiar with debugger and programming) encounter bugs while buliding PE (without logging), how can they get help?
So logging and debugging should be provided at same place.

1-2) Modules / Library
The more library the language has, the better.
I think this part is VBScript's weakness - langs like python has lots of mature libraries and package manager.

1-3) Huddle for newcomers
Nature of WB082 syntax is 'batch', so understanding the syntax was not that so hard.
(Of course, with the expense of limited power and messy syntax.)
Which lang will provide least huddle to newcomers?

1-4) Converting WB082's command model to function model
We have to think about compability with WB082 syntax, like providing some 'shims'.
For example, python does not have native api to read/write registry.
Can be done with FFI, so PEBakery should provide its own library to access registry.

2. Case of inventing new syntax

2-1) Reinvent of wheel?
Contemporary programming languages already have some clear design.
Which means if we invent new syntax, time is spent to reinvent wheel.

2-2) Batch-like syntax vs Programming lang-like syntax
As I said before, nature of WB082 syntax is 'batch'.
Which syntax should we orient?

Plugin container format is another thing we should improve too, like slore indicated.
I think it can be done indepentally with syntax, so let's talk ideas about it too.

PS. Current stage of implementation
Test Build (20170510) : * PEBakery_TestBuild_20170510.7z (2654.66 kB - downloaded 14 times.)

- Plugin Cache
Serialize plugin into database, and take it back when loading.
PEBakery's raw load time is very slow at first time.
(Win10PESE with Download folder has 1000+ plugins, PEBakery takes 30sec to load even with i7-7700HQ, multithreaded.)
But starting from second load, it takes only 3sec, thanks to caching.

Cons of caching is the posibillity of database's corruption.
PEBakery works with admin privileges, so it can cause security breach.
If internal plugin instance is changed in next version of PEBakery, it can also cause corruptions.

- Build Engine
Variable and Macro (aka API) system implemented.
PEBakery now supports tirivial subset of WB082 commands, you can try with clicking buttons.
Simple works like viewing messages, writing to text, etc. is now supported.
However, important commands like file and registry operation is not implemented yet.

- Code Optimizer
Remember stuttering at 7-zip's 'More Options' button?
It is due to repeated file I/O, which can be done at one time.
PEBakery now optimizes continued Visible, TXTAddLine experimentally.

- Log Viewer
Experimental log viewer is supported.
Try runing some button with build log viewer open - you will see logs are updated simultaneously.
I prefer to use 'Export to TXT' functionability when debugging, though.

- Setting
Several options are now supported.

Macro/Command Support / Re: Syntax error in Section [Download_Files]
« on: April 22, 2017, 12:39:13 AM »
I hadn't tried to write kind of 'converting' operations.

But since PEBakery generates kind of parse tree internally, I think cleaning nested If/Else would be able later.

I guess I misunderstood your question.

PEBakery currently supports nested If/Else, so there is no need to convert into single If.

However, if we can, converting nested If into single If will helps us in point of readability.
(It will also help converting WB syntax into new syntax)

Macro/Command Support / Syntax error in Section [Download_Files]
« on: April 21, 2017, 09:19:54 AM »
There is syntax error in Section [Download_Files].

Code: [Select]
// This section also used directly by ML
// 1 Folder 2 FileName 3 Web (Cancelled 4 NoExit )
If,#3-,Equal,-,Echo,"Download File - Parameter Missing%,Warn

One of the doublequote is missing in Echo,"Download File - Parameter Missing%,Warn.
I found this error while testing Macro Library with PEBakery, WinBuilder seems to ignore this.

To be sure, which environment (BIOS, UEFI) and architecture (x86, x64) are you using?

Those messages are intended for use with CD/DVD in BIOS mode, not USB stick.

It seems you flashed full ISO into USB using Rufus.
Instead, try using WriteMedia\Copy to USB-Device BCD BootMGR instead.
I guess it will fix your problem.

Win10PE SE HomePage / Re: Error report
« on: April 19, 2017, 03:46:53 AM »
Use v1607 (Redstone 1) ISO, it seems v1703 (Redstone 2) is not supported yet.

Hello, ZYX! Welcome to the forum.

1. It would be nice to have faster and more reliable builder compatible with all existing plugins for the start.

I agree with you.

2. I very like slore's idea with VBscript as a programing language for a new syntax.

I haven't used VBScript, what is the advantages of using VBScript's syntax?
Are there any customizable parser and runtime exists?
To support log functionability and others, I think customizing runtime is inevitable.

We also have to consider about easiness of converting WB082 syntax into new syntax.

3. I don't like .NET dependency, I would stay at least to 3.5 version included in Win7 if it is possible at all.

I understand about your opinion, but I think using C#/WPF with recent .Net Framework is the best choice.
I had also considered about using C++/Qt, but it did have some limitations compared with C#/WPF.

First, using C# shortens developing time.
Since I don't have much free time, longer developing time is burden to me.
Thinking about .Net Framework version, C# in .Net Framework 3.5 does lack useful features.
Moreover, starting from Windows 8, Microsoft pre-install .Net Framework 4.x, while 3.5 is not.
Thinking of Windows 7's support will be dropped around 2019, I think using .Net Framework 4.x is better.

Second, .Net Framework 4.6.2, supports long file name (up to 32K) out-of-the-box.
As you know, 255byte file length limit of WB082 sometimes wreck PE building.
Of course, supporting long file name can be done in native Windows API, but it is quite complex.

Third, WPF supports GPU-acclerated rendering and HiDPI support out-of-the-box.
WB082's layout rendering is too slow, you can try with clicking 'More Options' button in '7-zip' plugin.
Sadly, Per-Monitor HiDPI support is supported starting from .Net Framework 4.6.2 (and with Win 10 v1607)
In WPF, GPU acclearation and HiDPI support comes with no cost.

WinBuilder is slow and has memory problems. LiveSystem Pro is fast, and almost bugfree (after helping Kare remove countless bugs).
But the grammar (syntax) for both builders sucks big time. I'm tired of all those comma's and return values as arguments.

Why not write something that has a syntax like a normal language ? So I am writing this right now.
Lexer is done, parser is 95% done. It has full expression support. Syntax is sort of PureBasic like (which I'm using to write it in).

So if someone is writing a replacement builder, why not change the syntax as well. Sure, it will break everybodies projects including mine, but it also gives a lot of new and better commands.

Totally agree. The syntax of WB is also one of it's main problems.
LSP mostly inherit this style and soon there will be another builder with this silly synatx  :frusty:

I agree with Martt and JFX's opinion about WB's syntax, it is total mess.

Free Limited Time Evolution should go step by step since there is no programmer team that can transfer all projects and all plugins with new syntax.

However, we don't have time to convert current SE projects by hand, as Lancelot described.

This is what I am planning:
1) Implement WB082 compatible builder <- Current status
2) Propose new PEBakery file format and grammar
3) Write converter, and convert all plugin into new format

Process 2 and 3 will take years, but if all goes well, I hope bad legacy to be removed.

That is why I have process 2 and 3 in roadmap.
WB082 syntax support is needed for current SE projects' codes, while we need new, clean syntax for the future of SE projects.
To transition, we need to support old grammar, new grammar, and converter.

In my opinion, Martt's grammar seems intuitive, while easy to convert from WB082's syntax.
Martt, are you willing to merge our work?
I can provide GUI and WB082 syntax implementation, while you can provide new grammar and parser.
After our work becomes stable, we can convert all SE plugins into Martt's new grammar, and everyone will be happy.

Plugin interface rendering nearly done, here is test build: * (1199.57 kB - downloaded 32 times.)

- All interface control implemented (with the exception of CheckList)
- Button parsing and rendering improved
- More image type supported (jpg, png, gif, bmp, ico, svg)
     Limitation : White color is not considered as transparent, can be substituted by png with alpha channels.
- Clicking image (open URL or show with external viewer) implemented
- Wrong font point-to-pixel conversion fixed
     As a result, text layout is less malformed.

There was a problem that interface is too small, so in the Setting dialog, scale factor option was added.
(Currently this value is not saved so you have to set it every time)

Scale of 100%

Scale of 200%

Yet there is long way to go...
- WPF's image rendering system considers image's native DPI, causing plugin logo to be shrinked.
Ex) Win10PESE/Tweaks/My Computer - Name : Logo's DPI is not 96.
Solution = Change logo into strict 96DPIm or just ignore DPI.
(In fact, I tried ignoring DPI, but failed.)

- Scale factor system works nearly fine, with the exception of blurry image.
Solution = Must provide image of higher resolution or DPI.

Win10PE SE HomePage / Re: Update Plugin cannot download problems
« on: March 26, 2017, 08:24:04 AM »
In South Korea, download works well.

Code: [Select]
$ ping                                                                                                                                                                                                           
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=44 time=185 ms
64 bytes from icmp_seq=2 ttl=44 time=185 ms
64 bytes from icmp_seq=3 ttl=44 time=188 ms
64 bytes from icmp_seq=4 ttl=44 time=186 ms
64 bytes from icmp_seq=5 ttl=44 time=184 ms
64 bytes from icmp_seq=6 ttl=44 time=185 ms
64 bytes from icmp_seq=7 ttl=44 time=192 ms
64 bytes from icmp_seq=8 ttl=44 time=190 ms
64 bytes from icmp_seq=9 ttl=44 time=192 ms
64 bytes from icmp_seq=10 ttl=44 time=188 ms
64 bytes from icmp_seq=11 ttl=44 time=184 ms
64 bytes from icmp_seq=12 ttl=44 time=189 ms
64 bytes from icmp_seq=13 ttl=44 time=189 ms
64 bytes from icmp_seq=14 ttl=44 time=187 ms
--- ping statistics ---
14 packets transmitted, 14 received, 0% packet loss, time 13015ms
rtt min/avg/max/mdev = 184.940/187.952/192.945/2.664 ms

However, one of local ISP's foreign traffic is very unstable now for weeks, packet being dropped just like you.
I guess that there are some failures in East Asia - US internet connection?

Thanks for your opinion, Lancelot.

Your suggestions are vital, and here is some ideas about improvements:

1. Logging
To me most critical,
winbuilder use wrong kind of memory management,
 very wrong log file which cause big log and + with bad memory management cause winbuilder hang ...

For logging, I intend to use SQLite database.
This way, logging won't explode PEBakery, and provide more reliable and reconstructable log.

2. Lag, Lag and Lags
also winbuilder can not ignore encoded files during read of a plugin + bad memory management = cause slow down of big sized plugins during execution and during interface edits.
=> we use workaround with File Container plugins.
===> but initial execution of winbuilder cause too much time with increasing number of plugins following bad plugin management with bad memory management.

At my laptop, WB082 took 35sec to load, while Windows nagging like "No responce. Can I shut it down?".
So I put progress bar at PEBakery, no more Windows' nagging.

Currently PEBakery load only [Main], [Interface], [Variables], [Process], etc at load time.
Encoded file parts are lazy loaded at ondemand, and also utilize multicore for performance.
Because of this, PEBakery loads at 10sec / 100MB while WB082 took 35sec / 20MB to load.
Slower than 4sec and takes more mem than 20MB, but it seems to be successful tradeoff.

3. Wow64 redirection
I believe
can also be depricated.

Since PEBakery is C# application, it support x86 / x64 both with one binary.
System,FileRedirect can be safely removed.

4. Future of language specification
as long as you can decode, I do not think there will be problem for compatibility.
 during evolution lots of formats tested,
  at that time there were programmer contributers, but like project developers they were kicked out from development

It would be good to see a good working open source builder with a good evolution to better.
 Something we sadly miss and mostly workaround during development of projects.
 At least we designed projects and plugins in a way to be compatible to a new builder which will help to a new builder development.

This is what I am planning:
1) Implement WB082 compatible builder <- Current status
2) Propose new PEBakery file format and grammar
3) Write converter, and convert all plugin into new format

Process 2 and 3 will take years, but if all goes well, I hope bad legacy to be removed.

5. Tirivial things
- Fast UI rendering
Since WPF utilize GPU accleration, no more WB's slooooooooow UI rendering.
No more facepalm in while opening more options in '7-Zip File Manager' plugin.

- Unicode Support
If I use CJK characters in WB, WB just freezes.
PEBakery already supports UTF8 and UTF16.

WinBuilder do not support even System Aware DPI awareness.
Starting from Windows 10 v1607 and .Net Framework 4.6.2, WPF apps support Per Monitor DPI out-of-the-box.
The only problem is plugin embedded images - they need to be changed to higher resolution.

- Long file path (<32k)
We had problems with putting .Net Framework into PE because of 260B path restriction.
PEBakery support long file path up to 32765B out-of-the-box, powered by .Net Framework 4.6.2.

Sick up with WinBuilder 082's bugs and bugs, I am writing compatible builder, PEBakery.

The goal is to make WB082 drop-in replacement, and I expect the work to be completed in late 2017.
Here is test build : * (1196.43 kB - downloaded 27 times.)
(Need .Net Framework 4.6.2 to work)

Currently the program's UI part is being worked.
Since WB has lots of undocumented behaviors, I need some help.
I was confronted by these while researching about them:

1. Button control
According to WB document, button control has operands up to 11.
However, in production, it has various length of operands.
Code: [Select]
<11 operands format>

<12 operands format>
pButton_TopicLink="[TheOven] Topic",1,8,430,27,97,25,HelpTxt_TopicLink,0,True,_HelpTxt_TopicLink_,True

<14 operands format>
Button_CompareEdit=,1,8,426,-1,24,24,CompareEditXXX,AmperossQetto2Move0016016.bmp,False,False,_CompareEditXXX_,False,"__DOWNLOAD and Compare with an Editor"

I cannot figure out exact grammar.
'SectionName' and 'ProgressShow' are duplicated, format being too obfuscated.

Currently my implementation support only part of 12-operand-format:
Code: [Select]

As a result, lots of buttons are ommited in interfaces:

WinBuilder 082 rendered

Does anyone know exact grammar?

2. UI Controls to be deprecated
I cannot find usage of CheckList in Win10PESE.
If there is no demand for this, can this be deprecated?

Ideas about new UI control is also good.

3. Encoded File Format
I figured out WB082's encoded file format:
Code: [Select]
<Attachment Format>
Streams are encoded in base64 format.
Concat all lines into one long string, append '=', '==' or nothing according to length.
(Need '=' padding to be appended to be .Net acknowledgled base64 format)
Decode base64 encoded string to get binary, which follows these 2 types
[Type 1]
Zlib Compressed File
- Used in most file
- Base64 encoded string always start with 'eJ'
- Base64 decoded bytes always start with '78 9c' (in hex) - which is zlib stream's magic number

[Type 2]
Untouched File + Zlib Compressed Footer
- Used in already compressed file (Ex zip, 7z)
- Base64 decoded footer always start with '78 9c' (in hex) - which is zlib stream's magic number

Footer : 550Byte (Decompressed)
[Length of FileName]
Stream of mostly 0 and some bytes - Maybe hash? for integrity?

Fortunately, footer is not essential to extract attached file.
Because of unknown footer, writing PEBakery's own attach format will be much better in future.

[How to improve?]
Use LZMA instead of zlib, for better compression rate

The problem is Type 2.
It's 550Byte footer format is unknown, so I can extract file, but cannot encode.

Theses are sample of decompressed footer:
Original File : * Wind_x86_Origin.7z (2.39 kB - downloaded 16 times.)
Decoded with footer : * Wind_x86_WithFooter.7z (2.46 kB - downloaded 13 times.)
Decompressed footer : * (0.16 kB - downloaded 15 times.) (unzip this)
Original File 2 : * D2Coding-Ver1.1-TTC-20151103.7z (3056.85 kB - downloaded 17 times.)
Decompressed footer 2 : * (0.18 kB - downloaded 15 times.)  (unzip this)

4. Opinion about script language
Actual build had been implemented, but it was total mess.
Script engine must be fully refactored, so will take months.

This xlsx file shows PEBakery legacy engine's work state:
* (15.69 kB - downloaded 16 times.)

I think some commands should be deprecated or redesigned.
Is there any opinion about language improvement?
Any information and opinion about variables system is also helpful.

Win10PE SE HomePage / Re: Notepad tweaks
« on: December 14, 2016, 06:35:47 AM »
I found out recent version (v45) of notepad plugin contains bug.
It tries to call PCopy_notepadexeMui section which does not exist.

Code: [Select]

As a result, it emittes error.
This line should be removed or PCopy_notepadexeMui section must be implemented.

(I tried removing that line, and it works well, however)

Win10PE SE HomePage / Re: Remote Desktop Connection Bug in v1607
« on: September 01, 2016, 08:03:50 PM »
Sorry for late reply, I tested this plugin and it is working well with my Win 10 v1607 RDP server.
Thanks for your help.

Win10PE SE HomePage / Re: Imperfect Auto Colorization in Theme in v1607
« on: August 23, 2016, 07:16:02 AM »
I tested Color 1 (Auto), 7, and 13 with updated plugin v31.
Auto colorization (No1) is still malfunctioning, presenting pink afterglow again with skyblue wallpaper.
(In fact, that skyblue + flower wallpaper was extracted from Win 8 install media! So the chance of broken image format is not true).

The good news is, specifying theme color (No2-15) works well in updated plugin.
I think it is suitable to be uploaded.
Maybe adding this message will help the users:
Code: [Select]
If auto colorization malfunctions, try specifiying color manually

It seems afterglow color is disabled, using gray even though the color is specified in registry?
Since gray is harmonious with most colors, it doesn't matter for now.
(gray+blue is much better than pink+blue  :smile:)

Thanks you for updating plugin blazingly fast, ChrisR.

Win10PE SE HomePage / Imperfect Auto Colorization in Theme in v1607
« on: August 22, 2016, 08:15:21 AM »
In Win10PESE built with Win 10 v1607, Auto Colorization is working imperfectly.

In this picture, theme color (blue) is well picked, but afterglow color (pink!?) is not related with background image.

While Auto Colorization works well in v1511.

I found out these registrys affect Theme color, but these values are overrided in PE when AutoColorizaion is off...
Code: [Select]
HKCU\SOFTWARE\Microsoft\Windows\DWM\ColorizationAfterglow the comment implies.
Code: [Select]
// For testing Colors. it does not work for now with a grey title. No matter, AutoColorization is enough
RegWrite,HKLM,0x4,"Tmp_Default\Control Panel\Desktop",AutoColorization,0

I am researching about this phenomenon, but result is still failure.
As least adding 'Turn off auto colorization' option seems to be necessary (for anyone prefer gray to pink).

Thanks to ChrisR's quick fix, Logon as Administrator now works.
However, when Administrator session is initailized, an error appears.
It should be related to Logon as Administrator plugin, since it does not happen when built without Logon as Administrator.

(Translate: Unknown Software Exception (0x80000003) occured in executable at 0x00007FFDE087C4D7)

Strangely, it seems this error do not distrub real use of Administrator session.
When I switch sessions (Administrator -> SYSTEM -> Administrator), this error do not appear.
And I could launch userland executables without any problems.

I coded [UseAlternative_Process] to be processed under one of these conditions is met:

1. calc.exe is not provided while building Win10PESE
If these condition is met in original version, an error dialog will appear and WB skips calculator plugin.
I thought about copying PrecCalc regardless of %ProvideFilesOK% var, but it doesn't make sense.
Normal users (who can get Win 8.1's calc) would prefer Windows' default calc since it is more familiar.
So if Win 8.1's calc.exe is provided (== %ProvideFilesOK% var is set to 1), Windows' calc will be used.
Only in case of %ProvideFilesOK% is set to 0, PrecCalc will be used as fallback.

In code, in these case, %t% is set to 1.

So when testing condition 1, you should rename or delete 'MS_Calculator_NT6' folder.

2. 'Use PrecCalc Instead' option is checked
In fact, these option was used as kind of debug option while writing this plugin.
If these option is checked, PrecCalc is used instead of Windows' regardless of condition 1.

In code, in these case, %t% is set to 2.

When testing condition 2, you don't need to touch 'MS_Calculator_NT6' folder.

Win10PE SE HomePage / Remote Desktop Connection Bug in v1607
« on: August 22, 2016, 07:03:56 AM »
While testing Win10PESE built with Win 10 v1607, I am confronting some bugs.
One of them is Remote Desktop Connection (mstsc.exe).

When I try to connect my own RDP server, an error appears.
(Translate : Remote computer requires network level auth user's computer does not support)

My RDP server is Win 10 v1511 running on QEMU, and in my Laptop which runs Win 10 v1607, connection sucesses.

Process Monitor shows me mstsc.exe requires these 4 dlls, but even thougth they are added, same error appears.
Code: [Select]

Even Win10PESE built with v1511 has same situation.
Maybe these article ('s instruction will work?
I did not test these instructions yet.

Another thing I worry about is, v1607's mstsc uses 'modern' UI in password input dialog.
(Picture took from my host OS, Seems related to Windows Credential Manager)
But Win10PESE does not support modern UI yet as long as I know.
I wonder how these part can be handled?

In Win10PESE, calculator plugin needs Win 8.1's calculator to work properly, because Win10PESE does not support 'Modern App' version of calculator.
However, grapping 8.1's file manually takes time and is bothersome.
Distributing 8.1's calculator is prohibitted by EULA, so we cannot embed them.
So, why not use third-party distributable calculator instead?
(Just like Notepad plugin does with notepad2!)

Precise Calculator (PrecCalc) is free software and distributes official binary in 32bit and 64bit both.

Quote from: preccalc.chm

This program is distributed under the terms of "GNU General Public License". You can get it from the author's web page or from Here is only a short abstract of this license:

1) The Program is free. You may distribute it in any medium. There can be other programs (free or commercial) on the same medium.
According to its license, we can embed PrecCalc into plugin file.

Here is v21 of Calculator plugin:
* Calculator_v21.Script (552.03 kB - downloaded 62 times.)

I added 'CheckBox_UseAlternative' option, which substitute Windows's calc.exe with PrecCalc.
However, PrecCalc needs some folder structure to work.
So instead copying PrecClac directly to System32, I installed an 'hook':
1) Copy PrecClac just like other apps in 'Run from RAM' mode
2) Copy fake calc.exe, which acts like detour executable.
3) When fake calc.exe is launched in PE, it execute PrecCalc and quit.
4) PrecCalc will be opened, and everyone is happy.

If v21 detects provided files are not available in Win10PESE, 'CheckBox_UseAlternative' option is automatically enabled.

Fake calc.exe's source is on github :
It reads executable's path from 'PrecCalcPath.ini', and launch with ShellExecute() API.
It understands envrionment variable such as %SystemDrive% (simple WinAPI call does this).
It requires little dependency as you can simply ignore.
This is distributed under MIT License, you are free to use and modify source code.

For anyone who want to use PrecCalc and Windows' default calc.exe both, I wrote standalone plugin, too.
* PrecCalc.script (494.44 kB - downloaded 66 times.)

I hope it will smooth user experience when building PE.

Win10PE SE HomePage / Re: Korean IME Support for Win10PESE
« on: August 13, 2016, 03:57:09 AM »
v23 released.

I added some files to enable "Language" control panel.
The list of the file were taken from Chinese IME, thanks to sp-star's work.

It was tested on Win8.1SE and Win10PESE (v1607).

Pages: [1] 2 3 ... 6
Powered by EzPortal