39 Questions to Ask When Choosing Report Software
Introduction:
While choosing the wrong product is never a good idea, when it comes to clinical trials software, the wrong product could be disastrous for your company.
We've made a genuine, good will effort to compile the most important factors that should be considered before buying. Some questions are fairly basic, but most are insightful and thought provoking.
If you think of additional questions that should be asked by others in the decision mode, please pass them on to us and we'll include them in updated editions.
Q1. Will your software run in my existing computer environment?
Reason for asking:
You should first determine if your operating system is supported and your computers adequate.
DZS Answer:
The ClinPlus Report Engine can be installed under any operating system that supports SAS. Current installations include Windows and, Unix. The Report GUI is installed in a Windows environment. A hybrid installation is supported via SAS/CONNECT where the Report GUI resides under Windows and the Report Engine resides under Unix.
Q2. What programming language is your system written in?
Reason for asking:
To determine if the system complies with company standards and will easily integrate with existing systems and procedures.
DZS Answer:
ClinPlus Report is written in SAS, the industry standard in clinical data analysis. The current version is validated under SAS v9.1.3 and 9.2.
Q3. Is your system a collection of programs written to produce pre-defined reports or a tool used to create custom reports?
Reason for asking:
To ascertain overall system philosophy and to get a general impression of flexibility.
DZS Answer:
ClinPlus Report is not merely a report selector, rather a true report authoring tool. With this tool, users can create custom reports with the content and layout they require. These report templates can then be saved in libraries for future use. However, DZS does offer a complete library of safety table and listing templates that can be customized to your unique standards.
Q4. What type of reports can be created?
Reason for asking:
To determine the scope of the system. Ask about various statistical and summary tables, data listings, patient profiles, etc.
DZS Answer:
ClinPlus Report produces presentation quality statistical tables, data listings and patient profiles.
Q5. How is the system accessed, through programing, interactively (GUI), or both?
Reason for asking:
To get an idea of the range of skill sets required and integration possibilities.
DZS Answer:
The system is used either through an intuitive GUI interface or by programming. Furthermore, programs can be created using the GUI?
Q6. Who in my company will benefit from this system
Reason for asking:
If a wide variety of people like programmers, data managers, data coordinators, statisticians, QA and managers can see a benefit then the system will be more valuable and easily cost justifiable.
DZS Answer:
Statistical programmers and statisticians use ClinPlus Report to create professional looking, submission quality tables and listings in less time and with more consistent results.
Data managers are easily able to produce high quality data listings as required during an ongoing study. Patient profiles can also be created and used to facility the database QC process by selecting a random percentage of subject data to be printed. A QC summary report will provide record and data point counts along with error acceptability calculations.
Medical writers benefit from the high quality RTF tables that may be included in the NDA submission document without any reformatting at all. This will greatly reduce human error risks in the final steps of the process.
Stakeholders will also benefit by the time saved in preparing a superior submission document that will help bring the product to market sooner.
Q7. Will this system produce 100% of my reports?
Reason for asking:
To better understand the systems range and capabilities.
Inevitably you will come across a unique report requirement that can not be fulfilled by ClinPlus Report. However, our clients report that virtually 100% of safety tables and about 80-100% of efficacy tables are easily produced along with 100% of listings and patient profiles. DZS developers continually strive to improve system functionality and to meet ever-changing requirements and standards.
Q8. Is your software fully validated?
Reason for asking:
Software validation is a key requirement of clinical data systems used for FDA submissions.
DZS Answer:
DZS follows strict SOPs written to satisfy FDA and industry regulations and guidelines. No ClinPlus software is released until it has been fully validated.
Q9. Are validation documents available to me?
Reason for asking:
If a client or regulatory agency audits you, your software vendor should be able to produce a well organized complete validation report with supporting documentation and assist you with the audit.
DZS Answer:
Yes. A copy of the validation summary may be requested and all supporting documentation will be made available to you for an audit. DZS personnel are also available to assist in an audit. We also recommend that you conduct a vendor audit before investing in any clinical trials software.
Q10. do you provided a test plan?
Reason for asking:
This is important if your SOPs require that you validate the software in your environment. A comprehensive test plan may take many weeks, even months to write from scratch.
DZS Answer:
The test plan used in the validation process is provided along with any required data and programs. Test plans are extremely comprehensive and written in a cookbook style allowing persons not familiar with the software to conduct the tests.
Q11. What is involved in setting up a study in your system?
Reason for asking:
This may be an indicator of ease-of-use and flexibility.
DZS Answer:
Users need only specify directory locations for data, views, format catalogs, macro libraries, program libraries and output location. A Study is typically set up in 5 minutes or less.
Q12. Does my data structure need to conform to your system?
Reason for asking:
You'll want to know if your data standards will need to be altered and how much pre-processing is required to "fit" the data to the system. Ask about data set names, variable names, types, formats, sort order, etc
DZS Answer:
ClinPlus Report does not have specific requirements regarding data set structures, variable names, etc. However, data should be structured logically for a desired report. DZS offers a CDISC SDTM to ADaM conversion toolkit to automate and document production of analysis ready data.
Q13. Is your system data driven, that is , will the reports de formatted correctly regardless of the numberof across and down categories?
Reason for asking:
This will attest to the reusability of the programs across studies.
DZS Answer:
Yes, a ClinPlus Report program displaying statistics across treatment groups will produce a properly formatted table regardless of the number of groups
Q14. Does the system generate stand-alone programs?
Reason for asking:
Once a program is created, you'll want to know if it can be executed independently of the system
DZS Answer:
Stand-alone programs are easily generated from the GUI interface or can be created from scratch by a programmer. These may be modified and are submitted for processing like any other SAS program
Q15. Can programs created by your system be delivered to a third party?
Reason for asking:
You may be required to send programs to a government agency or the study sponsor. Make sure they can execute these programs without having to license the system.
DZS Answer:
This can be accomplished in two ways
- 1) A run-time catalog generator is included that creates a version of software that can be used only to re-produce the target reports. Run-time catalog users will be able to alter certain formatting options.
- 2) The actual code generated by the Report Engine can be saved and delivered. This eliminates the need to deliver the Report Engine catalog.
Q16. What output formats are produced?
Reason for asking:
Most clinical trials software programs are quite specific in their application. Be sure to determine exactly what applications are supplied as well as what other integrated modules are available. Although you many not recognize a current need for a specific capability, it is advisable that the software you choose be able to smoothly integrate with the full scope of clinical trials software requirements.
DZS Answer:
ClinPlus Report will directly create ASCII and RTF formats. ClinPlus's RTF engine provides much greater flexibility than SAS/ODS in producing output for custom tables.. RTF style sheets are used to specify parameters (fonts, colors, border styles, shading, etc.) for titles, footnotes, headings, report body and layout. PDF output is easily created from the RTF output using software such as Adobe Acrobat. HTML output is also supported through SAS/ODS.
Q17. Can intermediate output be generated and saved?
Reason for asking:
You may want to print output generated by the individual statistical procedures for review and documentation.
DZS Answer:
Yes, you can request default listings and/or ODS data sets created by SAS procedures to be produced.
Q18. How flexible are the report layouts? Can I control page size, line size, margins, row and column spacing, column widths, fonts, border styles, colors, etc.?
Reason for asking:
A well-designed system should be able to meet your company's current standards or the standards of various study sponsors and be flexible enough to adjust to future standards.
DZS Answer:
A wide array of options is included to control virtually every aspect of your report layout. In all likelihood, ClinPlus Report will enable you to produce report layouts that exactly or very nearly conform to your current company standards or the varying standards required by CROs
Q19. What if I have more categories than will fit on one page? How is page wrapping handled?
Reason for asking:
Find out if the system will logically break categories automatically. For instance, statistics pertaining to a treatment group will not be split across pages and see if you can explicitly set the break point. Also, find out if a wrapped report can be printed on a single page
DZS Answer:
ClinPlus Report features intelligent page breaking and overflow logic to ensure that groups will always be split at a logical point. You will never have N & Min, on one page and Max & Mean on the next for a variable. You may also explicitly set page break points, repeat key variables (such as CENTER and PATID) on the continued page and request an overflowed table be split and centered on a single page.
Q20. Can I control uniformity of categories displayed?
Reason for asking:
Let's say you want to show columns for visits 1 through 5 for each treatment group whether or not data exists in every visit. Can you do this? If this is the default action then can you suppress it?
DZS Answer:
Yes, uniformity is completely configurable at any grouping level.
Q21. Can the report orientation be easily changed? That is, if we decide we want statistics or variables running across the report rather than down, what's involved?
Reason for asking:
After producing a report, you may decide that a different orientation is more appropriate or, if you are reusing a program for a study with different requirements you may need to make this type of change. Find out how easily this is done, or if it can be done at all.
DZS Answer:
ClinPlus report uses two pairs (mutually exclusive) keyword parameters, STATSACROSS/ STATSDOWN and VARSACROSS/VARSDOWN to choose one of four basic table orientations. Just change this keyword combination (or select from a radio box control in the GUI) to choose the orientation you desire.
Q22. How are column labels produced?
Reason for asking:
Some systems will break a line mid-word. You will want to avoid this. Column label text should be able to be split, stacked and centered over the column, justified left or right, or indented as needed.
DZS Answer:
By default, labels are logically split at word breaks and stacked centered over the column. You may also specify explicit break points, alter justification (left, right or center) or indent each line n number of spaces.
Q23. Can a count (N=n) be included with the label?
Reason for asking:
You may want to include a count on certain columns such as treatment group, visit, etc. Ask if these counts can be calculated from a data set other than the primary input data set such as your demographics data set. Also ask if this display format can be altered.
DZS Answer:
Yes, these counts may be calculated from any data set and displayed in the format you desire
Q24. What is the flexibility of the report titles?
Reason for asking:
Find out if the system supports multiple justifications within a title line, inclusion of paging, analysis and system variables (date, time, page #s, etc.) and if counts (N=n) can be included with paging variables. Also, find out if titles be specified symbolically (macro variables) from a global titles library.
DZS Answer:
Yes, uniformity is completely configurable at any grouping level. You can also choose to show frequency counts for all format values whether or not data exists.
Q25. What is the flexibility of report footnotes?
Reason for asking:
Find out if "smart" footnotes are supported. That is, a footnote is present only when a condition exists on that page. Determine footnote placement options. Reports may have a better appearance when footnotes are snug to the bottom left corner of the report rather than on the bottom of the page. See if variables, like in report titles, can be included.
DZS Answer:
Smart footnote capabilities are amongst the newest features of ClinPlus Report. Footnotes can be left, right or center justified, indented n spaces or aligned with the left edge of the table. They may also be positioned at the bottom of the page or snug up to the bottom of the table. Paging variables may also be included in your footnotes. The number of footnotes is limited only by the physical page size. The RTF table places footnotes in table cells and may include table of contents tags.
Q26. How are page numbers printed?
Reason for asking:
You may want a format such as N of TOT or to be able to start at any number or print N (Last) on the last page.
DZS Answer:
Page Numbers are printed where and how you choose. N of TOT page numbering is supported as is N (Last). You can explicitly specify the starting page number or suppress them altogether. In the RTF table, titles and footnotes are placed in table cells rather than the header/footer area. This eliminates table page numbers being reassigned as document page numbers.
Q27. Can you easily specify spanning headers?
Reason for asking:
You should be able to span columns with text such as spanning all treatment groups with Treatment. You should also be able to suppress any default spanning headers
DZS Answer:
Any number of spanning headers can be requested to span the columns you choose.
Q28. What statistics are supported?
Reason for asking:
To be sure that required statistics, both descriptive and inferential, are available.
DZS Answer:
Currently ClinPlus Report includes statistics from the following SAS procedures: Means, Freq, Univariate, GLM, Mixed, Npar1way and Lifetest. All in all, over 95 statistics are supported. Additional statistics are regularly added at client's request.
Q29. Can I include statistics, or other values, that are not directly supported in my reports?
Reason for asking:
Regardless of the number of statistics supported, in rare cases you will need to produce a statistic that that is not supported. Make sure this is possible
DZS Answer:
Yes, a special statistic keyword (SHOW) is used to display any numeric variable on your table. Simply calculate your statistic, include it on the input data set and SHOW it on the table.
Q30. Can I modify the statistical procedure's options and parameters?
Reason for asking:
Yes, a special statistic keyword (SHOW) is used to display any numeric variable on your table. Simply calculate your statistic, include it on the input data set and SHOW it on the table.
DZS Answer:
ClinPlus Report honors keyword parameters that control individual SAS procedure options and parameters.
Q31. For inferential statistics, can star notation be displayed for significance?
Reason for asking:
This is standard in most clinical reports. Also, check other options that are available for displaying significance.
DZS Answer:
ClinPlus Report will automatically display standard star notation for significance by request. You may also choose to display NS for Non-Significant values or suppress the star notation for p-values between, 0.1 and 0.05. An RTF style parameter allows you to automatically bold significant p-values within the range you specify.
Q32. Are columns displaying various statistics decimal point aligned?
Reason for asking:
Some products simply right justify numeric values. Decimal point alignment is the proper method of display
DZS Answer:
ClinPlus Report columns are always decimal point aligned.
Q33. Can I explicitly and implicitly specify rounding for statistics?
Reason for asking:
You will want to be able to explicitly set rounding to whole numbers, tenths, etc. and, at times be able to round statistics relative to the incoming precision. For instance, if you always display Means at 2 decimal places beyond the incoming precision then you may want to be able to implicitly set the rounding as such so that rounding will vary from study to study as incoming precision varies.
DZS Answer:
ClinPlus Report supports both implicit and explicit rounding.
Q34. Can two statistics share a column?
Reason for asking:
You may want to display statistics like Min and Max as a range, side-by-side or display a percentage next to a count in a formatted manner.
DZS Answer:
Yes, you may also define custom formatting like (min - max) or concatenate variables such as CENTER and PATID (010001). You may also display percentages as 100%, (100), etc. or you can choose to add any characters before, between and/or after statistics
Can multiple variables share a column in a listing?
Reason for asking:
Providing listings that fit all required data elements for a subject within a single page gratly improves readability. Find out if multiple variables can be stacked and wrapped within a single column.
DZS Answer:
Yes, you can choose to stack up to 3 variables within a column with each layer indented as you specify. You can also specify the column width so values will be wrapped (on word boundaries) within the column. Subject data is never spilt across pages.
Q36. What options are there for calculating and displaying percentages?
Reason for asking:
Find out if you can specify a data set and variable to be used as the denominator and how to handle missing values. And, if you can include special display options such as <1% for values that round to 0, suppression of printing 0%, etc..
DZS Answer:
Many options are provided to control the calculation and display of percentages including handeling of missing values, denominator data set/variable control, displaying <1% for values that round to 0, suppression of printing 0% and suppression of printing 0%.
Q37. I add totals and sub-totals for columns and rows?
Reason for asking:
This is a must for any report generating system. Ask what options are available for this.
DZS Answer:
Yes, column and/or row totals can be included at any grouping level. Also, frequency totals can be added such as Male, Female, Total. You can also choose to display Total as the first or last line in a group.
Q38. Can I alter the default sort orders?
Reason for asking:
The alphanumeric order of values is not always the true logical order. You may also want to display a variable by it's formatted values rather than it's raw value or vice-versa.
DZS Answer:
have full control over the sort order of variables in your tables and listings.
Q39. What user documentation is available?
Reason for asking:
Ask to see the User guide, training guide and on-line documentation. It should be complete, well organized and easy to understand.
DZS Answer:
Over the last several years, DZS has invested in completely re-writing its user documentation. The results are fully indexed, professionally written user guides and training manuals. An HTML based version of the user guide is accessible through the ClinPlus system. All clients receive as many hard copies as they need plus MS Word and PDF versions on CD along with the integrated HTML version.
Q40. What does your software cost?
Reason for asking:
Keep in mind that the price of the software is but one line item in determining system cost. Other things to consider are: hardware costs, third party software licenses, implementation charges, training charges, additional support staff required (IT, DBAs, programmers, data management staff, etc.) and ongoing maintenance charges. Before deciding, find out how the software is priced (per seat, per site, average concurrent user, blocks of users). You may also wish to explore available payment options and trial periods.
DZS Answer:
DZS offers a one-time license fee, a leasing program or can design a payment program that will fit your unique requirements. The leasing program includes maintenance (tech support and all upgrades) while a separate maintenance agreement is required for the one-time license.
You will also need SAS licenses (Base and Stat) for each user. You may contact a ClinPlus sales representative toll free at 1-866-CLINPLUS or email us at sales@clinplus.com to discuss your requirements and options.
Prepared by
DZS Software Solutions
ClinPlus® Clinical Trials Software
Contact DZS Today!
