Visualization

For designing VisTra’s visual interface we closely collaborated with three domain experts, all researchers working on novel diagnosis methods. Our collaborators stated that it is invaluable to clearly present timing information along with transitive chains (R-3). For representation pur- poses, two of them suggested using message sequence charts as they are already very common in early development phases (R-10) and as they provide a good opportunity to show communication correlations over time. We therefore abstained from our initial idea of using a node-link diagram to represent the dependencies and rather explored how we could trim message sequence charts to our needs in terms of additional information and scalability.

Based on iterative refinement with our three domain collaborators, we came up with a vertically separated dual-view visualization (see Figure 5.4) where the left view was designed to provide

an overview over the physical network with ECUs and bus systems (ECU-Map View) and the

right view sequentially showed transitive chains in the requested message sequence chart-like representation (Sequence Chart View). To explore a specific transitive chain the user can select a FB, we call it the trigger-FB, and in doing so initiates a color-coded presentation in the ECU- Map View, and an update of the Sequence Chart View with the transitive chain around the selected trigger-FB. The design of the two views can be described as follows:

Views The ECU-Map View is a mixture of a common topology map (cf. Section 3.4.2, R- 10) and a treemap (cf. Section 2.3, R-2) initialized with the two-level hierarchy of bus system and ECUs (segmentation of columns and lines) and the number of FBs to assign the aspect ratios. According to common topology maps the central gateway has a specific position and is located at the very top of the ECU-Map and ECUs of the same bus systems are grouped in

Figure 5.4:Screenshot of VisTra showing the ECU-Map View on the left and the Sequence Chart View on the right. Within the Sequence Chart View, three ECUs are zoomed in, one connection is highlighted via mouse-over, and additional detail is shown in a semitransparent tooltip.

columns. The size of each ECU rectangle codes the number of FBs contained and therefore provides an implicit engineers’ metric for ECUs’ importance. In order not to abandon our size coding approach, ECUs with multiple bus connections are not presented redundantly but are connected to the most corresponded bus system, i. e., over which they sent the most signals. Additionally each ECU rectangle has a “status bar” to indicate the number of active FBs within an ECU, i. e., FBs that actively communicated in the trace that is shown. The status bar which is represented as a semitransparent greenish overlay on the gray ECU rectangles is fully included in the treemap approach so that the area of active FBs directly correlates with the entire system. The user can click on an ECU rectangle which in turn displays an alphabetical list of the included active FBs (and also—grayed out—inactive FBs for context retaining reasons). After a trigger- FB has been selected, both views are updated. The ECU-Map thereafter highlights all ECUs involved in the triggered transitive chain by changing their color and doing so allows for quickly detecting all physical components involved reachable from the trigger-FB. The selected ECU turns into a light yellow and gets an additional header in a more saturated yellow to label the

time line from top to bottom, the (filtered) signal communication between FBs is presented by horizontal, temporally-ordered “communication lines” between the respective FBs. The resulting visualization is a color-coded sequence chart with a temporal representation of the triggered transitive chain ordered by causality (Figure 5.4, right view).

Interaction The most important user activity in VisTra is exploring FBs’ transitive chains. For this purpose, the user can either click on ECU rectangles in the ECU-Map, or can directly select elements in the Sequence Chart View (FB-labels, -rectangles, -dots) in order to navigate to interrelated chains. We integrated a browser-like history, which allows the user to easily nav- igate back and forward in the transitive chains s/he explored so far. In order to provide a better orientation in highly complex transitive chains, we highlight the communication connections by enlarging it when the user hovers over it. Additionally, hovering over elements reveals detailed information about sender and receiver FBs/ECUs, exact timing information and exchanged sig- nals in a semi-transparent text box. Besides, we added a semantic focus and context zoom to the Sequence Chart View. This solved the problem that some transitive chains are extremely com- plex and too large for showing them properly on the available horizontal space. If a triggered transitive chain exceeds a certain size (we initially defined 25 FBs, but this can interactively be adopted by the user), all ECU rectangles in the sequence chart except the ECU, which contains the trigger-FB, are semantically downscaled and the incorporated FBs are hidden. To explore these FBs, the user just clicks on an ECU to expand the downscaled ECU to the entire functional view. Clicking again collapses the ECU respectively (cf.Figure 5.4).

5.2.3 Evaluation

In the following, I provide qualitative feedback that we gathered during think-aloud studies with eight domain experts, five focus group workshops with 3–8 engineers and from various informal discussions with prospect end users. In the one-hour long think-aloud studies we encouraged our participants to bring along their own traces to analyze current problems of their daily work. Five of them used this opportunity, for the remaining three we used an example trace prepared by us and let them conduct a set of predefined tasks. By conducting all these studies, we wanted to get experts’ estimations on our approach in terms of domain utility and potentials, but also evaluate understandability, usability and current restrictions of both visualization and data preparation technique. Especially, understanding the limitations helped us very much in building later tools for trace analysis.

judged to be easy understandable and usable. While in general the dual-view concept was also judged positively, by observing our participants using the tool, however, revealed that the ECU- Map was rarely used compared to the Sequence Chart View. With the sequence chart, especially the path highlighting via hovering and the semantic zoom was liked by our participants such as one engineer cited: ‘‘oh good, this [path highlighting] is practical, by using it I do not have to follow the lines with my fingertip”, and another: ‘‘opening and closing of the ECUs is enormously helpful and understandable, especially when I have two ECUs in mind that I want to explore in more depth”. A frequent point of critique was the color coding in terms of using saturated colors for large areas. While this originally was an artifact of the participatory design process we con- ducted, we definitely agree with our participants (and also with literature, e. g., Ware [War08]). Furthermore, two of the automotive engineers mentioned that representing DTCs (errors) with VisTra would benefit in order to directly correlate dependencies with errors that have been oc- curred. Another missing feature frequently mentioned by engineers was filtering transitive chains in terms of time and shown levels of dependencies.

Data Preparation, Potentials and Limitations In addition to this, we intensively dis- cussed our participant’s estimations about the prospected utility but also the limitations of our approach and in doing so gathered feedback and suggestions about what to address with future tools.

Regarding the potentials of VisTra, all participants named the opportunity to better understand and faster track down complex errors by providing a novel perspective on the data (R-2). Fur- thermore, two trace analysts stated that especially“bundling the signals”is helpful as redundant information is automatically filtered in order to provide a more comprehensive view on depen- dencies in the network (R-1, R-6). Another two engineers underlined the importance of showing both time and logic in parallel and evaluated VisTra as a “first good step into this direction”

(R-3). While we generally got good feedback regarding our visual interface and data preparation, we, however, also learned about various technical limitations our first approach had and why these restrictions impeded a practical application in daily practices. These limitations strongly influenced the recommendations we introduced above and could be clustered in three categories, additional time costs/repetitive work (R-8, R-9), additional views/raw data (R-4, R-5, R-9) and personal preferences (cf. diversity implication inSection 3.6.2).

Additional time costs: This was the most important limitation mentioned by nearly all partic- ipants. VisTra is a standalone tool that uses exported trace files with a specific format (.asc), can handle traces with a maximum size of approximately 100MB, and additionally requires BNE exports for pre-computing dependencies. This hinders engineers using our tool in many ways. First, if a trace in question is not delivered in the correct data format it must be transformed be- forehand. Second, if traces are too large (i. e., >100MB, what they often are) they have to be split and explored separately in the tool. Third, finding and exporting the right network configuration takes time and can be a potential source of mistakes. Last, as VisTra is a standalone that is not

from different test cars showing the same behavior, or to refer to other traces in order to consult references). Manual and repeated preparation of all required traces is impractical (R-8). For using VisTra or other novel trace analysis tools in every day work, therefore all our participants argued for closely integrating them with current software environments. One engineer formulated it this way:“The most important thing for us [FIT analysts] is to get a tool that supports our work. For this purpose it is crucial to integrate it in Carmen [the analysis environment most frequently used by his group] otherwise we cannot benefit from any new tool [...] there have been so many good ideas, but all of them just had been nice examples and we never could use one single of these tools productively”(R-9).

Additional views/raw data: More than 90% of our participants furthermore noted that VisTra as it stands cannot be used for daily work because it lacks in providing additional perspectives on the data. Cited by nearly all engineers was a textual representation of either the interpreted trace or the raw data (R-5), as one engineer stated: “This list is the trace! We need it all the time for quickly referencing back”, or another one: “every visualization is a potential source of errors, therefore I always need the trace [in textual form] to check my hypotheses with the real data”. During our studies we also found that participants demanded other special views for specific needs (R-4). Two engineers said they would need a view for additionally showing dtc files, another three argued that a signal plot would be helpful, and one said he would need an additional view for showing vehicle status. While it might be too time- and cost-intensive to re-implement all these views, a complete and close integration of a tool in prevalent software environments would implicitly provide these additional views (R-9).

Personal preferences:In line with the demands for highly task- and user-specific views, our par- ticipants mentioned a variety of other requirements which they stated to be crucial for them, such as constantly highlighting cyclic messages, or showing the sequence chart horizontally rather than vertically. These aspects, however, rather indicate personal preferences and specific needs of a single participant or smaller sub-groups than providing generalizable implications. Asking other participants often revealed that for them a particular aspect or an additional view (except the view for the raw data!) would not be necessary, should be waived, or even solved differently (cf. diversity implication inSection 3.6.2).

5.2.4 Discussion

VisTra, the prototype presented in this section, is based on pre-computation and -filtering of traces utilizing dependency specifications from earlier development phases. Compared to tra- ditional message based analysis techniques, VisTra allows the user to explore traces starting

with trace analysis experts including think-aloud studies, focus groups and informal discussions. While the basic approach was quite liked, the studies revealed several obstacles that would hin- der VisTra’s usage in daily practices. These findings helped us in better understanding domain requirements and especially in formulating the recommendations R-4, R-5, R-8 and R-9 (cf.Sec- tion 5.1.4). First, we learned that repetitive work and unnecessary iterations is a major reason for not using a solution (R-8). For practical application, novel tools need to be closely integrated with prevalent analysis environments (R-9). We also found that it is invaluable for nearly all engineers to have fast access to a textual representation of the trace in question (R-5) and also that other traditional perspectives on the data would be beneficial (R-4, R-9). Finally, we en- countered a variety of specific needs differing between sub-groups and even single engineers (cf. diversity implication inSection 3.6.2). Usually however, it is neither possible nor advisable to address all these personal, even conflicting requirements at once in a novel tool design. Based on our experience, we think and recommend to start designing novel tools by focusing on a specific target group with similar needs and requirements, and to engage them in a participatory design approach. While we strongly that it is valuable to have a broad understanding of different fields in mind when designing the tool (i. e., doing pre-design studies with various user groups), during designing phase it might be easier and more practicable to have clear requirements from a specific user group. If this sub-group has carefully be chosen in terms of representativeness and a design has validated for them, in a next step the tool could be tested with other sub-groups and adapted to their specific needs.