This section d iscusses the issues addressed in developing the communications and networking services needed to support the Learning Shell.
The req u i re ments for an e-learning system (Table 4 . 2 ) demand that students be able to exchange messages within the cou rse, upd ate their course materials as needed, and receive up-to-date learn ing s upport. Tutors, on the other hand, must be able to com m u n icate with students a nd update learning materials and the support database as req uired. All these tasks requ i re communication over the network. Some IMMEDIATE features, such as Extra mural Support, also req uire behind-the-scenes commun ication across the network at the system leve l .
I n t h e network specification for I M M E D IATE (Fig ure 5 . 9 ) , each Learning Shell connects to the Course Repository via a Communications Manager, using custom protocols on top of FTP. In the implemented version the Commun ications Manager has been broken d own fu rthe r into two components - an FTP Server (Appendix H 1 ) and a Repository Manager (Appendix H2) - wh ich have been installed together with the Repository on a single computer.
The i mplementation a ssumes that the Repository would be managed by an experienced systems ad ministrator, who would set up new users, complete some tasks semi-manually, a nd optimise settings as required . Repository management services operate at two levels - the FTP Server a nd the Repository Manager.
Chapter 6 : Prototyping 1 5 1
The FTP server acts as the point of entry into the I M M E D IATE network. lt logs the user onto the network, assigns them to their correct folders , and allots them the permissions to complete the necessary file transfers. To support this, two separate user profi les have been created in the server - a tutor and a student. All tutors a re assigned a single profile and share a "Teachers" folder system. Each student shares the same profile, but is a lso allotted their own root directory (Figure 6 . 23).
U niversity Machine
C drive
VLM_Repository
Repository
Archive - Resou rces Students Database
'"'L
LooL
Updates StudentNL
Learning Comp1 Learning Comp2 Learning CompX:::::,,,0,
CD_Only - Resources U pdates Teachers1
:
ec:::
geBox Updates Working_ Files MessageQueue Temp U pdatesArchiveL
VLM Learning Comp1 Learning Comp2 Learning CompYt=
Help Updates ResourcesL
Learning Comp1 Learning Comp2 Learning CompZFigure 6.23: Repository folder structu re.
The FTP server used is War_FTP, developed by Jarle Aase and freely downloadable from his web site . 1
For the I M M E D IATE network to be accessible, both the FTP Server and Repository Manager must be online at all times. Once launched , they remain ru nning in the backg round in a minimised state , appearing as icons in the Windows task bar. The system ad ministrator double-clicks on these icons to bring the a pplications back into view.
Chapter 6 : Prototyping 1 52
6 . 6 . 1 Repository Manager
The Repository Manager i m plements those aspects of communications and data tra nsfers unique to I M M E D IATE . lt ena bles d ata on users' machines to be synchronised and updated across the network, wh ile allowing for simultaneous access by m u lti ple users, and for the possibility that the connection with any or all users may be broken at any time d u ring the process. To meet the req uirements of the Learning Shell three kinds of data transfers a re managed : a synchronous electronic messaging between course participants, period ic updating of the Extramural Support system on each student's computer, and the uploading of updated learning resou rces. This has req u i red :
• a n appropriate Repository structure;
• a set of com munication protocols, implemented over the TCP/IP protoco l fa mily which ensure accu rate synchron isation between sender and receiver;
• a message-queuing mechanism, which ensures that the earliest message received
is the first processed;
• a parsing mechanism for reading/writing messages from/to a database; and
• a separate set of p rotocols for updating learn i ng resources on each student's mach ine.
A detailed , daily log is provided to enable the admi n istrator to monitor network traffic and troubleshoot problems, a nd optimise settings. The main screen of the Repository Manager can be seen in Figure 6 .24. Its underlying architecture is shown in Fig u re 6 . 2 5 . Selected source code is attached as Appendix H 3 .
Repository
The Repository is a set of folders with the structure o utlined in Figure 6.23 . The folders conta in an arch ive of the e ntire course content including a master copy of the Learning Shell System Database; a l l commun ications amongst those involved in the course; u pd ated course materials for the students to upload ; and individual folder syste ms for each student, including a mailbox for new message s and space for the Learning Shell to store backups of the student's work if desired .
Chapter 6 : Prototyping
o6ilory
rf!anager
Stop Server Settings t.lcssagos OK Hulp Update Zip Resources'1:: ;:/ .::._ :i;:fl.l?' llf\OISEd t/O'•IC�t 1nl?f\.'O SE(Of•JS ' ( ::7 42 Scr."C· �Jr:t-cc
. t �U .:::_ _,SI r'i?Cft,Ed ro?(f .€SihC•..,.• tot '( JD .c;:: \to !lt.rf:d rrc�}!IQC:O"'' 'Bcb' · t jU .:::. �io -;zc•�d n �?:.;.,(�-; ·o· 'Bel)'
'( 30 ¥.: _!I!; I rcceM:d rcqY.:�p·c:cc:.: c d l c r Bo:r :c 1:: ·?� 00 CC DJ
· t -�u .:::.
: \V'J.t =.C':lc:IIJr:/•.=-c:ro: tcr_.·•. tc::;o3c-)uc-...ct.l;a .. 8Bt·T
., JU !� - � �>?C
' ( J 1 �:: \le:o:b3c u�-j�e rcct.c:"'C� t:/ Bcb
' I:: Jl ,_._ . NIJ '•:I' ))mj:ll?"l?•,pc.J"'? ?plsror-,"'' )ffl bel) �-er
u:;:;ell l.lc::o;c T:iPC ) [I &-1
'I: �1 J n I?S.;.�:u:l'?S ·::et? ·.�?j ju l::jc� ' ( ) I (;: 'to �tcred rrc:o:.!IQ:.:i "'' •o-�·
' I; �1 J n Bs :.)�S luYriJE""·td SoM!r SRhrp El ' ( 31 ::: :>ce1ng •. \'-/J.1_HE'O(r;I1)1,\:..,I?:J(I!.I(t':•,) '( 31 c::: :·c�c Clear urwbg P
Figure 6.24: The Repository Manager.
Main Screen
System Adm i nistrator m anages the repository from here. Module implements
synchronisation prototcols for m anaging commun ications.
Database
Mod u le Screen
Access Module
Server Viewer Prepares G uides
Customising Window to view updated manages administrator
access to the
monitoring messages to extram u ral
Repository through various
intervals, log adm inistrator & support tables
database by repository
visibility, etc all messages in for
other management
database transmission
components tasks
across network
Course archive, updates, system database, mailboxes
Figure 6.25: Repository Manager arch itecture.
Chapter 6: P rototyping 1 54
6 .6 . 2 Asy n c h ronous messag ing
T o meet I M M E D IATE 's specification , asynchronous electronic messaging has been i m plemented u sing a message-q ueuing techn ique that automatically handles com m u nications from m u ltiple users on a first-in, first-out basis. These messages may be between course pa rticipa nts, between the tutor and the system administrator, between system components and the tutor or administrator, or between constituent applications at the system-to-system leve l .
Only t h e ad m i n i strator can view all mail held in the database.
Messaging protocols
A set of com m u nication protocols had to be designed and implemented at both the client a nd server end, to ensure the integ rity of communications ove r an unreliable network, to which mu ltiple users may be connected concurrently. These protocols operate on top of the File Transfer Protocol . Messages are aggregated i nto a single file for transmission , which is parsed into i ndividual messages by the receiving application . The message tra n sfer process entails a two-way exchange of messages between the u ser and the Repository.
Beca use messages a re stored in relational tables they must preserve the constraints on a replicated database distributed over multiple computers:
• Each m essage is unique within a single copy of the database so as to avoid
redundancy or a primary key conflict.
• Duplication of messages between d ifferent copies of the database must be
possible .
• Copies of all m essages connected with a particular user m u st be stored on that use r's com puter. But all messages do n ot need to be stored on a l l computers.
At a minimum the protocols had to ensure that :
• any message lost through a network interruption will be re-transmitted the next
time that u se r con nects;
• tra nsmissions from multiple users can be interwoven without difficulty; a nd
• no message will be transferred twice .
A set of protocols has been designed to meet these requirements (Figure 6 . 26).
Chapter 6 : Prototyping 1 55
d iscussed next, proved sufficient to meet these requ i re ments.
Central to the protocols is a two-step synchronisation process to ensure that a l l the messages that need to be sent, and only those that need to be sent, are transmitted from each end of the connection, while allowing for overlapping commun ications from d ifferent users (Figure 6 . 27).
Message-queuing
Message-queuing applications commun icate by the client passing a message to an intermediate structure in which incoming requests are queued until the server is ready and able to deal with the m . Any response from the server is retu rned via a similar mechanism. Through the prototyping process, the following mechanisms were determined for implementing message-queuing in I M M E DIATE .
Client-to-server commun ications are hand led th rough a Repository folder named 'MessageQueue' to which all users are assigned access. All i ncom ing commun ications a re placed in this folder, which is monitored by the Repository Manager at a regular i nterval. The mon itoring interval is determ ined by a timer component (thread ) to which a checkMsgQueue method is attached . The administrator can adjust th is interval .
Server-to-client commun ications are handled through an individual 'MessageBox' folder located in each user's Repository root directory .
The MessageQueue folder is organ ised as a priority queue of messages in the following man ner:
1 . At the end of each monitoring interval , the checkMsgQueue method is ca lled . The
files in the MessageQueue folder are sorted by name. These file names will look
like '67888.txt' or '67864.txt'.
2 . The first item, which will be the one with the earliest date_time stamp (smal lest integer), is processed.
3 . Once processed, the response is named either 'answer. txt' or ' messages.txt' in line with the messaging protocols, and placed in the user's MessageBox. This naming convention ensures that any older versions of these responses inadvertently left behind will be overwritten by the new ones.
Chapter 6: Prototyping 1 56
Precondition:
The user has successfully established a connection via the FTP server.
Message Format:
• All transmissions are in the form of text files, in which the first five lines constitute a header which must follow a set format as below.
<TO RCVR> Who the transm ission is addressed to. Must be
the actual receiver of the transm ission , and be the
System , the Tutor or a name on the class list
<FROM SNDR> Who has sent the transmission. Must be the
System, the Tutor or a name on the class list.
< M ESSAGE TYPE> May be a last received request/answer, a l ist of messages or a test transmission
<GROUP NUMBER> The Work G roup to which the user belongs. -1
means the tutor. The defau lt g roup is 0.
< DATETI ME STAMP O F LAST Default = 0.0, which means send all messages for
TRANSMISSION SNDR RECEIVED the sender i n the repository database
FROM RCVR>
• Each message within the body of a communication is separated by a new line contai ning only the
stri ng '[END]'.
• A transm ission from a client application is named with the string form of an integer representing the date and tim e that the transmission was composed a nd sent, as in '<date_time_as_no> .txt'.
• A transmission from the Repository Manager will be nam ed either 'answer.txt' or 'messages.txt' i n
l ine with t h e steps i n t h e handshake protocol
Message Transfer
Messages m ust be updated via a two-step synchronisation and transfer process:
• The client application sends a transmission to the Repository requesting the date-time stamp of
the l ast m essage successfully transferred from the client to the Repository.
• The Repository sends a transmission to the client containing the date-tim e stamp of the last
m essage successfully transferred from the client to the Repository. This message must be named 'answer.txt'.
• The client sends a transm ission containing the date-time stamp of the last m essage successfully
transferred to the client from the Repository, and all the outgoing messages with a date-time stam p later than the date-time received from the Repository.
• The Repository sends the client a transmission containing all the i ncoming messages for that client having a date-time stamp later than the date-tim e received from the client. This message will be named 'm essages.txt'.
Chapter 6: Prototyping Student or Tutor User Date_ Time
i
I
D"' T" "''" Synchron1sat1on Repository ManagerFigure 6.27 (a): Two-step transmission protocol.
.........
..
- -
··· ···
-- -.....
. . ...... .
··········
·············:······
�........
..
....
.
. .
..
..
.···
········
-- - - -- --
-.
······ ···
--
- - - - - -··· .
.....
- -··· ···
...
- - - " -- - - - - :;....- -- A B Date_Time Student or Tutor Users Repository Manager 1 57Chapter 6: Prototyping 1 58
From the user end:
1 . The Learning Shell checks its Message Box in the Repository at regular intervals for the required file.
2. When the file is found it is moved to the user's machine and deleted from the server.
3 . I f no file arrives within a specified time period , or the connection i s broken , then the tra nsaction is aborted by the Learn i ng Shell and the user is notified and requested to try again later.
File-Parsing
All tra n smissions within the I M M E D IATE system are text files in which the first five l ines constitute a header and the rema inder the body, as defined in Figure 6 . 26 . These files m ust be com posed at one end of the transmission and parsed at the other. The code for achieving this at the Repository Manager end of transmissions is included in Append ix H3.
When the Repository Manager finds a file in the Message Queue end , it passes it to the processMsgList p rocedure . The processing steps taken by this procedure are :
1 . Process the transmission header
2 . I f transmission is a message list, then
a . Update the Repository with the messages
b . Return a list o f messages t o sender based o n Last_ Received information provided 3. Else if transmission is a Last_ Received request. then
a . Return Last_Received information to sender
P rocessing the header is sim ply a matter of parsing the first five lines of the file for the necessary info rmation .
Messages are stored in a relational database on each computer within the IMMEDIATE n etwork. They are not transm itted as i nd ividual text files but as a list of messages each sepa rated by an '[E N D]' symbol . This list includes all the outgoing messages in the sender's d atabase that have not already been successfully added to the recipient's d atabase.
All messages include a date_time stamp, recording when they were composed (i.e. the " Entered" field in the Messaging table in Figure 6 . 1 9) . A Last_Received request from a client, v iz. a Learning Shell o r authoring applicatio n , is fulfilled by the Repository finding
Chapter 6 : Prototyping 1 59
the incoming message from that client with the most recent date_ti m e stamp, using a n SOL query l ike:
select MAX( entered ) from Messaging where F _From = : F _From ;
where the ' : Fro m ' variable takes the value of the client user n a me. And the user responds with a Last Received message to the Repository Manager based on the res u lt of:
select MAX( entered ) from Messaging where F _From <> : F _From ; where ' : F _From' again takes the value of the client user name.
To compose a message list, the sende r uses the Last_Received data to run an SOL q uery on the local copy of the Messag ing table (Fig u re 6.28). The result set is then written to a text file, with each field written to a new line and ' [ E N D]' separating the messages. Updating the Messaging table at the receiver end is just a matter of reversing this process.
(a) Query run at client end : select *
from Messaging
where ( F _From =: F _From or MsgType = "Q") and entered> :entered ;
(b) Query run at Repository end : select *
from Messaging
where entered> : entered and F _From <> :F _From and( F _To = : F _To or Copy To = "All"
or ( CopyTo = "MyGroup" and G roupNo =:groupNo ) ) ;
Figure 6.28: SQL q ueries for composing message lists to be tra nsm itted.
However, the main body ("Text" field in the Messaging table in Fig ure 6. 1 9) of a message may be many (or zero) lines long . Thus, the primary role of the ' [ E N D]' symbol is to mark the end of the Text field.
An advantage of transmitting messages as straight text files is that they are platform independent. lt is not necessary that the Repository and Learn i ng Shell databases h ave the same physical implementation . lt is only necessary that they have the same logica l structu re.
The key methods for implementing the messaging system at the Repository end are shown in Appendix H3. Im plementation at the Learning Shell end is similar. The interface for the Shell messag ing m od u le is included as Appendix H 6 .
Chapter 6: Prototyping 1 60
6.6.3 Proced u res for Resou rce U pdati n g ove r the network