Multi-Threading using xlsout
Hey Guys, Long story short I am trying to extract results into excel quicker and am trying to multi-thread this process. However it seems that even though the multi-thread side of things works within my script only 1 file will properly extract and the rest will just be empty workbooks.
Process Process-54:
Traceback (most recent call last):
File "C:\Python27\lib\multiprocessing\process.py", line 267, in _bootstrap
self.run()
File "C:\Python27\lib\multiprocessing\process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "C:\Dropbox\Power System Services\Scripts\New folder\PYTHON\2.7\PSSE Python Scripts\PSSE\PSSE_MAT\DynDataExtractorMATMulti.py", line 179, in run_data_extract
outfile=outfile1, sheet='', overwritesheet=True);
File ".\dyntools.py", line 1084, in xlsout
File ".\dyntools.py", line 428, in _xls_export
File ".\excelpy.py", line 168, in <module>
File "C:\Python27\lib\site-packages\win32com\client\__init__.py", line 11, in <module>
import gencache
File "C:\Python27\lib\site-packages\win32com\client\gencache.py", line 662, in <module>
__init__()
File "C:\Python27\lib\site-packages\win32com\client\gencache.py", line 56, in __init__
_LoadDicts()
File "C:\Python27\lib\site-packages\win32com\client\gencache.py", line 109, in _LoadDicts
version = p.load()
EOFError
This is the error code that comes out. This is essentially the only line of the code which is causing these errors.
dyntools.CHNF(outfile1).xlsout(channels=signals, show=False, xlsfile=(PathDir + "\\sheets\\" + filename + str(fileNumber) + '.xlsx'),
outfile='', sheet='', overwritesheet=True);
Note this code works as intended if I reduce the thread count to 1 however that sort of defeats the point. Cheers
EDIT Thanks for the response EBahr. - I implemented your method of multi-threading the process and still recieving an error once I increase from 1 thread to any other number.
Traceback (most recent call last):
File "C:/Dropbox/Power System Services/Scripts/New folder/PYTHON/2.7/PSSE Python Scripts/PSSE/PSSE_MAT/ExtractionPSSE.py", line 137, in <module>
main()
File "C:/Dropbox/Power System Services/Scripts/New folder/PYTHON/2.7/PSSE Python Scripts/PSSE/PSSE_MAT/ExtractionPSSE.py", line 115, in main
results = out_pool.map(build_xls, processes)
File "C:\Python27\lib\multiprocessing\pool.py", line 253, in map
return self.map_async(func, iterable, chunksize).get()
File "C:\Python27\lib\multiprocessing\pool.py", line 572, in get
raise self._value
pywintypes.com_error: (-2147418111, 'Call was rejected by callee.', None, None)
Below is my implementation it simply requires a set of Out files named like so. (Please excuse my lack of structure :/) Results_0.out
Results_1.out
Results_2.out
Results_3.out
import sys
import os
import shutil
import Tkinter, tkFileDialog
import ntpath
import csv
import time
import multiprocessing
from multiprocessing import Pool as ThreadPool
import math
PSSE33BINPATH = r"C:/Program Files (x86)/PTI/PSSEUniversity33/pssbin;C:/Program Files (x86)/PTI/PSSEUniversity33/psslib;"
def main():
# Prompt user for input
root = Tkinter.Tk()
root.withdraw()
resultType = "Teebar"
dirname = tkFileDialog.askopenfilename(parent=root, initialdir="/",
title='Please select a _0.out File')
PathDir = os.path.dirname(dirname)
PathDir = os.path.abspath(os.path.join(PathDir, os.pardir))
if not ...
Your code works as intended for me. I wonder if you are running into a memory limitation or have an issue with you .out files?
Interesting, did you switch back to psse v33 ? Also how would I tell if I am hitting a memory limitation, I think my .out files are fine otherwise it would not work individually
I switched to PSSE V34 and was able to get it to work with 2 threads, however it failed at more than this.
Hi, I am trying to extract results into excel, the code That I use is datos_excel = dyntools.CHNF(archivo_out + '.out') #datos_excel.xlsout(channels='', show=False) But theese error is displayed Server busy, this action can not be completed because the other program is busy, Choos
Hey Juan, I just posted an answer to your question. If you have some example code if you could add it to that question I would be happy to have a look at it.