Enterprise Wide Schedulers vs EDI Translator Scheduling Tools
Some organizations (especially Fortune 1000 companies) chose to use enterprise level job scheduling software (aka Workload automation) for all IT jobs not just EDI. The enterprise scheduling software is maintained by the IT operations team and includes dozens of other jobs in addition to EDI such as mainframe jobs, SAP jobs, PeopleSoft jobs and etc. The scheduling software has adapters installed on the servers where the jobs are actually executed and they run as a service. Then the operations team uses a client version of the software to set up the jobs and monitor them. Some examples of popular job schedulers (in no particular order) are Control-M, Tidal Enterprise Scheduler, BMC CONTROL-M, and IBM Tivoli Workload Scheduler. There are a also half-a-dozen open-source job schedulers out there as well.
Pros: These schedulers have a lot of sophisticated functions such as dynamic calendars (e.g. Holiday Calendar, Year-End Calendar) and an incredible amount of dependencies such as waiting for a job or job-step to complete to trigger another event. Other dependencies include file-arrival and timing of file existence. A classic example of how the dependency helps an EDI job is for example an SAP (or any ERP) inventory replenishment job must be completed before the EDI 850 Outbound job can start. The completion of the ERP inventory replenishment job triggers the EDI job to start, therefore eliminating the need for a separate scheduled job for the EDI translation.
They usually have built-in tabs for run-books and instructions on the fly that they can provide to the operator for troubleshooting. (e.g. run-book for EDI translation job failing would instructor the operator to route a ticket to the EDI team).
Other features that usually come out of the box are FTP/SFTP clients with scripting capability, http web service calls and requests, cross-operating support. These software packages are usually operating system independent and work with most industry standard user authentication methods such as LDAP. Some of these packages have out of the box email and alert notification functionality.
The enterprise schedulers are also very Sarbanes-Oxley (SOX) and ITIL friendly. Most public companies treat EDI as a SOX application and the EDI team sometimes complies with SOX by having to use one of these schedulers.
Cons: Very expensive although sometimes worth justifying putting in effort to build custom dependency scripts. The costs range from under $10k to $100k plus, almost the cost of some EDI translators alone.
These products are usually heavily regulated by corporate IT policies which require strict change control and migration processes and approvals. The same scheduled job has to be created in three environments such as Dev, QA, and PROD and if a slight oversight occurred during the initial creation of a fully cycle change control process has to occur all over again. There have been scenarios where a job is created with a dependency on file 851.txt being in a certain directory. However, the implementer had actually intended the file to be called 850.txt. In this case the change had to occur in all three environments (DEV, QA, Prod) all over again after the change control board had approved the change (in this case the actual change was 1 byte in a file name). One may argue that the root cause of this disadvantage is the IT policy not the actual software and that may be a fair argument. However, such software packages bring with them this natural inclination to create mindless IT change control policies.
The only way that these enterprise level applications can execute jobs is by using the operating system’s command line / shell either by executing a batch file or a script (e.g. shell, Perl and etc) or actually executing the application. Some EDI software packages cannot be executed from a shell / command line (certain versions of GIS/Sterling Integrator for example) and can only be executed within the scheduling system of the packages. For the EDI applications that actually support execution of jobs from the command line, one of the following usually ends up happening (in no particular order):
- EDI team ends up having to write a script file that kicks off the EDI translator’s executable file with certain complex parameters. These parameters are not usually in the software installation guide.
- EDI team ends up having to create a job in the EDI scheduler anyway (without actually scheduling it) and calls the EDI translator with the name of the job as the command line.
- Ends up having some of its jobs in enterprise scheduling and still has some jobs in the EDI scheduler because of certain nuances or restrictions that prohibit the job from existing in the enterprise scheduler hence making the EDI team support two scheduling interfaces.
EDI Translator Schedulers
Most commercial of the shelf EDI translation packages come with a built-in scheduling tool. Scheduling tools that come out of the box in translators are naturally more tightly integrated with translators and are more EDI oriented.
These schedulers have some basic dependency criteria selections such as a file dependency and transaction arrival dependencies.
PROS: There is no need to install additional client-side applications for the EDI team to monitor jobs. Since they are tightly integrated to EDI transactions pinpointing the exact issue that caused a job to fail or hang is easier. There is no need to write scripts that kick off the EDI jobs. EDI team members can also get to the issues quicker than having to have a ticket routed to them from the operations team. There is usually less red-tape with getting to fix issue. For example one common task for failed or stuck jobs is to simply re-start it. Since the enterprise scheduler will be usually be under harsher IT security controls
CONS: There is no centralized monitoring of the jobs 24/7. In this case, night-time operations team would have to have access to the EDI translator in order to monitor jobs there. There is a lack of interdependency with other applications.