logo
Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
EOD source that comes with multiple directories. - EOD source that comes with multiple directories.
Options
Go to last post Go to first unread
Previous Topic Next Topic
MetaStockDB  
#1 Posted : Monday, December 10, 2012 6:39:16 AM(UTC)
MetaStockDB

Rank: Newbie

Groups: Registered, Registered Users
Joined: 12/22/2011(UTC)
Posts: 6
Hi All,

I have EOD source that is formed by multiple "txt" files whereby each "txt" file represents a Sector of business such as Trading, Technology, Consumer, Property etc. Inside each "txt" file, it consists of symbols for that particular business sector.

These symbols are in files like "trading.txt", "technology.txt", "consumer.txt", "property.txt" etc.

I just put down the first 3 lines of for instance "trading.txt":
<TICKER>,<PER>,<DTYYYYMMDD>,<OPEN>,<HIGH>,<LOW>,<CLOSE>,<VOL>,<OPENINT>
ASB,D,20121210,0.1500,0.1500,0.1500,0.1500,1410,0
AEON,D,20121210,12.0000,12.1600,11.9000,12.1200,1772,0

The rest of the text files are same in format, carrying different symbols in their respective sector.

Each "txt" file, after conversion to MetaStock file format, will form a directory that consist of the symbols in the "txt" file. These symbols are now represented by "DAT" and "DOP" files.

This creates trouble for me to run "The Explorer" because if I would want to do so, I need to repeat "The Explorer" for each directory. Further more, I could not have a single report that tells me all the BUY/SELL signals at a glance.

Could someone help to provide me a script/batch file that could let the MetaStock's Downloader to Convert each symbols in every directory and save it in a SINGLE directory, so that I just need to run The Explorer once to filter through ALL the symbols?

Thank you very much.

Best regards.
Back to top

Wanna join the discussion?! Login to your Discussions forum accountor Register a new forum account.
Back to top  
jjstein  
#2 Posted : Monday, December 10, 2012 12:21:12 PM(UTC)
jjstein

Rank: Advanced Member

Groups: Registered, Registered Users, Subscribers
Joined: 5/13/2005(UTC)
Posts: 715
Location: Midwest, USA

Was thanked: 1 time(s) in 1 post(s)
MetaStockDB wrote:
This creates trouble for me to run "The Explorer" because if I would want to do so, I need to repeat "The Explorer" for each directory. Further more, I could not have a single report that tells me all the BUY/SELL signals at a glance.
Did you try to use the SECURITIES button, when you EDIT the Exploration?

MetaStockDB wrote:
Could someone help to provide me a script/batch file that could let the MetaStock's Downloader to Convert each symbols in every directory and save it in a SINGLE directory, so that I just need to run The Explorer once to filter through ALL the symbols?
If you already have a process in place to create the data files, it might be easier to make a simple batch file to re-COPY everything to a single directory, after processing.

It might help to know more -- what is your "EOD source"?
Back to top
wabbit  
#3 Posted : Monday, December 10, 2012 3:56:43 PM(UTC)
wabbit

Rank: Advanced Member

Groups: Registered, Registered Users, Subscribers, Unverified Users
Joined: 10/28/2004(UTC)
Posts: 3,111
Location: Perth, Western Australia

Was thanked: 16 time(s) in 16 post(s)
Once the data has been converted, it's going to be harder to copy into a complete "all securities" folder than if it is done before hand.

Strip off the first line of each of the files (the header row is not required) using the MORE command and then amalgamate all of the stocks by sector into one file using the COPY command, then pass this file to the Downloader for conversion. A batch file will mean the process is easily repeated each day.


wabbit [:D]
Back to top
wabbit  
#4 Posted : Monday, December 10, 2012 4:49:44 PM(UTC)
wabbit

Rank: Advanced Member

Groups: Registered, Registered Users, Subscribers, Unverified Users
Joined: 10/28/2004(UTC)
Posts: 3,111
Location: Perth, Western Australia

Was thanked: 16 time(s) in 16 post(s)
Just thought of something else to be wary of: If your data text files don't have a carriage return at the very end then problems can occur during the copy merge routine, so echo a new line onto the end of each of the text files before the copy merge, and then clean up any multiple carriage returns with a simple strfind after the copy merge.


wabbit [:D]
Back to top
jjstein  
#5 Posted : Monday, December 10, 2012 5:00:50 PM(UTC)
jjstein

Rank: Advanced Member

Groups: Registered, Registered Users, Subscribers
Joined: 5/13/2005(UTC)
Posts: 715
Location: Midwest, USA

Was thanked: 1 time(s) in 1 post(s)
wabbit wrote:
Once the data has been converted, it's going to be harder to copy into a complete "all securities" folder than if it is done before hand.
Oh, nuts! I forgot about Master/Emaster...a simple COPY won't work.
Back to top
MetaStockDB  
#6 Posted : Wednesday, December 12, 2012 5:23:55 AM(UTC)
MetaStockDB

Rank: Newbie

Groups: Registered, Registered Users
Joined: 12/22/2011(UTC)
Posts: 6
Hi Wabbit,

Thanks.

I managed to write simple codes to merge the files.
This is my codes.
------------------------------------------------------------------------
@ECHO OFF
cd ALLSYMBOLS
for %%f in (allsymbols-bak.txt) do del allsymbols-bak.txt
for %%f in (allsymbols.txt) do ren allsymbols.txt allsymbols-bak.txt
cd ..
for %%f in (*.txt) do MORE /E /S +1 %%f >> ALLSYMBOLS\allsymbols.txt
--------------------------------------------------------------------------

I need your guidance for writing batch commands that interact with "THE DOWNLOADER" so that I can automate the entire process.
Or I'd appreciate if you could enlighten me with any web site that provide guidance for such programming, then I could pick up the knowledge and write my own codes.

Thank you.

Best regards.
Back to top
wabbit  
#7 Posted : Wednesday, December 12, 2012 6:51:31 AM(UTC)
wabbit

Rank: Advanced Member

Groups: Registered, Registered Users, Subscribers, Unverified Users
Joined: 10/28/2004(UTC)
Posts: 3,111
Location: Perth, Western Australia

Was thanked: 16 time(s) in 16 post(s)
Google windows macros Wabbit [:D]
Back to top
MetaStockDB  
#8 Posted : Friday, December 14, 2012 9:59:14 PM(UTC)
MetaStockDB

Rank: Newbie

Groups: Registered, Registered Users
Joined: 12/22/2011(UTC)
Posts: 6
Hi Wabbit,

I need to bother you again. :)

Item 1. I have done this:
(i) Manually, by using The DownLoader, I am able to convert the historical EOD for past 20+ years in Multiple directories (in MetaStock format) into Multiple "txt" files.

(ii) Then I use "TYPE" command to append ALL these mulitple "txt" files into a SINGLE huge ".txt" file which contain the 20+ years EOD in txt format.
Note: FYI, "MOVE" command does not work well for big file I try to append because in the background (batch mode) it will need a space bar to continue to scroll the content output on the screen, and my destination text file (the SINGLE "txt" file) will capture the "---40% More---" kind of message and in fact the batch process is halted in background, waiting for space bar input and I could not provide it.
Although with "TYPE", I will capture the first line of every single "txt" file (<TICKER>,<PER>..), I found that The DownLoader seems to treat it as comment/remark as it does not complain, am I correct?
Anyway to avoid this first line of each "txt" file by using "TYPE"?

(iii) I use The DownLoader to convert this SINGLE "txt" file historical EOD into a SINGLE directory which contain MetaStock format EOD for ALL symbols for 20+ years.

(iv) So currently I can append the daily EOD (in txt format and in multiple text files) everyday by using The DownLoader to covert each and every daily EOD txt file, and append to the MetaStock format single directory.
I do this by highlighting all the daily EOD txt file in the source directory, select ASCII TEXT format, then choose the single MetaStock format directory as destination directory, and set destination files as MetaStock format.
My question need to consult you is:
There is error message saying that they are some ERRORS encountered during the above process BUT I open the charts and check through them, everything seems to be fine.
How do I know what type of error The DownLoader was complaining?
It seems that there is no detailed report can be found.
Do I need to take care any parameter in the Options for the Destination in the Conversion Options?
I just set:
(a) Append data to end of file. Replace matching dates.
(b) Include Open and Open Interest (Frankly I do not know what this is about).

Item 2. I have downloaded the AutoHotKey freeware for the Windows macro development. I am not sure if I am using the right tool. Kindly advise.

Thank you very much.

Best regards.
Back to top
wabbit  
#9 Posted : Thursday, December 20, 2012 5:33:12 PM(UTC)
wabbit

Rank: Advanced Member

Groups: Registered, Registered Users, Subscribers, Unverified Users
Joined: 10/28/2004(UTC)
Posts: 3,111
Location: Perth, Western Australia

Was thanked: 16 time(s) in 16 post(s)
As I dont have visibility of your system, your data files and exactly what you're trying to achieve...

You can use the Downloader to consolidate data files without having to consolidate the data into a single text file first. Just point the Downloader to the source directory and use the *.txt wildcard to gather all the text file source data and slam it all into the destination folder. Once the historical data has been added, just once, you only need to append the daily updates (a process which can be automated using any of the windows macro apps)


wabbit [:D]
Back to top
Users browsing this topic
Guest (Hidden)
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

Powered by YAF.NET | YAF.NET © 2003-2024, Yet Another Forum.NET