Liere. An environment for computational steering

JarkeJ.vanWijk12RobertvanLiere2

August6,1996

Abstract

ComputationalSteeringistheultimategoalofinteractivesimulation:researcherschangeparametersoftheirsimulationandimmediatelyreceivefeedbackontheeffect.Wepresentageneralandflexibleenvironmentforcomputationalsteering.Withinthisenvironmentaresearchercaneasilydevelopuserinterfacesand2-Dvisualizationsofhissimulation.Directmanipulationissupported,andtherequiredchangesofthesimulationareminimal.ThearchitectureoftheenvironmentisbasedonaDataManagerthattakescareofcentralizeddatastorageandeventnotification,andsatellitesthatproduceandvisualizedata.Oneofthesesatellitesisagraphicstooltodefineauserinterfaceinteractivelyandtovisualizethedata.ThecentralconcepthereistheParametrizedGraphicsObject:aninterfaceisbuiltupfromgraphicsobjectswhosepropertiesarefunctionsofdataintheDataManager.Thescopeofthesetoolsisnotlimitedtocomputationalsteering,butextendstomanyotherapplicationdomains.

Keywords:Visualization,computationalsteering,interaction,directmanipulation.

1Introduction

1.1ComputationalSteering

ScientificVisualizationhasbecomeamajorresearchareasince1987,whentheinfluentialreportoftheNSF[1]waspublished.Inrecentyearsmanynewmethods,techniques,andpackageshavebeendeveloped.Mostofthesedevelopmentsarelimitedtopost-processingofdata-sets.Usuallytheassumptionismadethatalldataisgeneratedfirstandthatnexttheresearcheriteratesthroughtheremainingstepsofthevisualizationpipeline(selection,filtering,mapping,andrendering)toachieveinsightinthegenerateddata.Hence,theinteractionwiththesimulationislimited.

Marshalletal.[2]distinguishtwoalternativestothispost-processingapproachtovisualization.Thefirststeptomoreinteractionwiththesimulationiswithtracking.Aftereachtime-stepofthesimulationtheresultingdataforthattime-stepissentintothevisualizationpipelineandcanbeinspected.Iftheresearcherconsiderstheresultsinvalid,thenthesimulationcanbestoppedatanearlystage,andrestartedwithadifferentsetofinputparameters.Thenextstep,Computational

Steeringgoesalotfurther,andcanbeconsideredastheultimategoalofinteractivecomputing.Computationalsteeringenablestheresearchertochangeparametersofthesimulationwhilethesimulationisprogressing.AccordingtoMarshalletal.[2]:\"Interactionwiththecomputationalmodelandtheresultinggraphicsdisplayisfundamentalinscientificvisualization.Steeringenhancesproductivitybygreatlyreducingthetimebetweenchangestomodelparametersandtheviewingoftheresults.\"

OuraimistoprovideresearcherswithaComputationalSteeringEnvironment(CSE)thatencouragesexploratoryinvestigationbytheresearcherofhissimulation.Inthefollowingsub-sectionswesumuptherequirements,wediscussexistingsolutions,andgiveanoverviewoftheremainderofthispaper,inwhichwepresentoursolution.

1.2Requirements

Thedata-flowbetweentheresearcherandhissimulationviaaCSEisshowninfigure1.Theresearchercanenternewvaluesforparameters,andviewsvisualizationsoftheresultingdata.Hence,inputwidgetssuchastext-fields,sliders,andbuttonsmustbeprovided,aswellasavarietyofvisualizationmethods,suchasgraphs,text,graphicsobjects,etc.Withsuchobjectsinputandoutputareseparated.Formoredirectcontrol,objectsmustbeprovidedthatallowfortwo-waycommunication:bothinputandoutput.Itmustbepossibletoselectanddragvisualizationobjects,therebydirectlycontrollingparametersofthesimulation.Inotherwords,directmanipulationmustbeprovided.

ThesimulationreceivesfromtheCSEnewparametervalues,andsendsnewlycalculatedresultstotheCSE.Weassumethatthesimulationcanhandlechangesofparametersonthefly,andthatitcanprovidemeaningfulintermediateresultswithinatime-intervalthatisacceptabletotheresearcher.TheconceptofdirectmanipulationhasacounterpartintheinterfacebetweenCSEandsimulation.Somevariables,typicallystate-variables,arecontinuouslyupdatedbythesimulation,butcanalsobechangedfromoutsideofthesimulation.

ResearchergraphicalinputdirectmanipulationanimationComputationalSteeringEnvironmentstatevariablesSimulationparametersresultsFigure1:Dataflowbetweenresearcher,CSE,andsimulation

Asanexampleofaninterfacewithahighdegreeofinteraction,considerfigure2.Thisshowsaninteractivegraphicsinterfacetoasimulationofasetofbouncingballsintwodimensions.Theballsaredepictedascircles,eachwithasmallredlinethatindicatesthedirectionandmagnitude

2

ofitsvelocity.Foreverytimestepthesimulationcalculatesthepositionandvelocityofallballs.Variouscontrol-parameterssuchasthesizeoftheballs,damping,attraction,aconstantfield-force,canbeset.Thiscanbedoneviasliders,bytyping,orbydraggingthearrows.Thepositionsoftheballs(state-variables)arecontinuouslyupdatedbythesimulation,butcanalsobechangedbytheresearcherbydraggingtheirgraphicalrepresentationstootherpositions.Thisfigureisascreen-dumpofaresultoftheCSEpresentedinthispaper.Insection4.1wecomebacktothisexampleandshowhowitwasdefined.

Radiusr0.125Contact forceFcd2500Contact dampingFcdvelocity0.000DampingFdvelocityTimeSynchronoust = 47.882dt = 0.005AsynchronousField force0.125-0.20Ekin = 0.06340AttractionFa0.000.005Figure2:Simulationofbouncingballs

Theresearchermustbeenabledtocreateandtorefinetheinterfacetothesimulationeasilyandincrementally.Theprocessofachievinginsightviasimulationisanincrementalprocess.Forallstagesofthevisualizationpipeline(fromsimulationtorendering)thecyclespecification,implementation,applicationiscontinuouslyreiterated.Intheprocessofgaininginsightviacomputationalsteering,aresearchertypicallywantstolookatandtocontrolother,possiblynewvariables,andtovisualizethemindifferentways.

TheCSEmustbeabletodealwithmultipleprocessessimultaneously.Therearethreereasonsforthis:First,itmustbepossibletointegrateexistingtools,suchasspecialpurposepackagesforgrid-editingorvisualization,intotheCSE.Second,asimulationisoftenbuiltupofseveralprocessesrunningondistantcomputeservers.Third,itwouldbeconvenientifseveralresearcherscouldviewandcontrolthedatasimultaneously.

ThefinalrequirementconcernstheunderlyingdatamodelandtheamountofdatawithintheCSE.Thetypeofdatatobehandleddependsverymuchonthetypeofsimulation,andthereforecanvaryfromsimplescalardatatolarge,three-dimensional,time-dependentvectorandtensorfielddata-sets.Theunderlyingdatamodelmustbeflexibleenoughtosupportawiderangeofdatatypesanddataquantities.

3

1.3Relatedwork

Anoptimalresultfortheresearchercanbeachievedifwebuildacompletesystem,incorporatinghandlingofinput,output,aswellasthesimulation,usingbasicgraphicslibraries,suchasforinstanceIRISGL,orPHIGS.Typicalexamplesareflight-simulatorsandpackagesformechanicalengineering.Thisapproachrequiresalargeeffortandconsiderableexperience,andoftenleadstoaninflexiblesystemwithfixedfunctionality.Thus,therequirementforeasymodificationbytheresearcherisviolated.However,iftheapplicationistime-criticalandhaswide-spreaduse,thegaincanbeworththecost.

Giventhesetofrequirements,howcanwerealizeamoregeneralsolution,i.e.anenvironmentratherthanaturn-keysystem?Basically,threeapproachescanbetaken:

considerthedesignandimplementationofaCSEasagraphicsproblem,andextendabasicgraphicslibrary;

consideritasauserinterfaceproblemandextendthefunctionalityofaUserInterfaceManagementSystem(UIMS);

consideritprimarilyasavisualizationproblem,andextendanapplicationbuilder.

Alltheseapproacheshavebeenpursued.Anexhaustivetreatmentwouldrequirefarmorespacethenisavailablehere,becauseverymanyrelatedapproaches,techniquesandconceptsexist.Welimitourselvesthereforetosomerelevantexamples.

Graphicslibrariestypicallyofferonlylowlevelfunctionality.Tosimplifythedefinitionofinteractiveapplicationswithdirectmanipulation,graphicslibrariescanbeenhancedwithhigherlevelinteractiveobjects.Withthisapproachatightcouplingofapplicationobjectsandinteractionobjectscanbeensured.Forexample,theInventortoolkit[3]providesanobject-orientedlibrarywhichsimplifiesthedevelopmentofinteractivegraphicsapplications.VanDametal.[4]takethisastepfurther.Theydescribeanextensiblesystemthatprimarilyaimsattheintegrationofavarietyofsimulationandanimationconceptsinthegraphicstoolkit.Objectsmayhavegeometric,algorithmic,orinteractiveproperties.Objectsmaysendmessagestoeachotherwhich,afterbeingstoredintheobject,canbeedited.Theauthorsshowhowcollisiondetection[5],constraintsanddeformation[6]arehandledwithinthisframework.

VariousUIMS’shavebeendescribedthatprovidesupportforcouplingtheapplication(repre-sentedbyasetofapplicationobjects)andtheuserinterface[7,8,9].UserinterfaceandapplicationdesignerscandeveloptheirpartsindependentlyandhavetheUIMSmanagethedialoguelayertointegratethetwoparts.SomeUIMS’sallowuserstospecifydirectmanipulationinterfacesthroughWYSIWYGeditors.AnexampleisthePeridotsystem[10],whichallowsnon-programmerstocreatesophisticatedinteractiontechniques.Withthehelpofgraphicalconstraints,Peridotallowsausertodrawgraphicalobjectsoftheuserinterface.Theuserprovidesthebehaviorofthesegraphicalobjectswithatechniquecalledprogrammingbyexample.ThisresultsinPeridotgener-atingparameterizedprocedureswhichcan,inturn,belinkedinorinterpretedbytheapplicationprogram.

Applicationbuildershaveemergedasaflexiblesolutionforscientificvisualization.Theusersareprovidedwithasetofmodules,whichcanbeconnectedandextendedtorapidlyprototypeapplicationsandreconfigureexistingones.Someresearchershavediscussedtheuseofcomputationalsteering[11,12,2]inthiscontext.Withthecurrentgenerationsofsuchsystemstheusercandefineuserinterfacepanels.Thesimulationhastobeincludedasamodule.The

4

extensionofsuchsystemswithdirectmanipulationhasbeendiscussedin[13].Ingeneral,theimplementationofdirectmanipulationiscumbersome,becauseindataflowenvironmentstherelationbetweentheoriginaldataandthegeometricobjectsinthevisualizationisnotknown.

Finally,anovelapproachtoexploratoryvisualizationhasbeentakeninVIEW[14].ThekeyideaofVIEWisaverytightcouplingofgeometrywithanunderlyingdatabase.TheVIEWsystemallowsresearcherstointeractdirectlywiththevisualization.Researcherscanselecttoolsforthevisualizationoftheirdata.Thisallowsandencouragesresearcherstoexperimentandexploretheunderlyingdataspaces.Scriptinglanguagesareusedfordefiningnewtools.

Allapproachesdiscussedhaveavalueontheirownright.However,wefeelthatnonesatisfiesallCSErequirementssimultaneously.Theextensionsdiscussedofbasicgraphicslibrariesareveryconvenientforgraphicsapplicationprogrammers,butarenotdirectlysuitableforusebyresearchers.ThevisualizationofdataisoutsidethescopeofcurrentUIMSs.Applicationbuildingenvironmentsdonotprovidedirectmanipulation,whichweconsiderasanimportantissueforcomputationalsteering.IntheVIEWsystemnewvisualizationtoolsmustbespecifiedviaaspecializedscriptinglanguage.Herethegraphicsobjects,andtherelationsbetweengraphicsobjects,data,anduseractionsarespecified.

1.4Overview

InthefollowingsectionswepresentanenvironmentforComputationalSteeringthatdoessatisfyourrequirements.Thespaceavailableheredoesnotallowforadetailedtreatment,amoreextensivedescriptioncanbefoundin[15].Insection2wegiveanoverviewofthearchitecture.ThecentralcomponentisaDataManager,whichisresponsibleformanagingdatastorageandprocesscommunication.Insection3ageneraltoolforgraphicalinputandoutputisdescribed.Wehavetakentheapproachofextendinglowlevelgraphics.WeusethenotionofaParametrizedGraphicsObject(PGO)asthemainconcept,andagraphicseditorasametaphorforthedesignoftheinterface.Examplesofapplicationsaregiveninsection4,followedbyadiscussion(section5)andconclusions(section6).

2Architecture

Flexibilityimpliesthatthefunctionalityisspreadoverseveralseparateprocesses,forsimulation,forinput,output,aswellasauxiliaryoperationsondata.Theseprocessesmusthaveaccesstothesamesetofdata.Wethereforeusethearchitectureshowninfigure3.ThecentralprocessistheDataManager,theotherprocesseswecallsatellites.SatellitescanconnecttoandcommunicatewiththeDataManager.

ThepurposeoftheDataManageristwofold:

Manageadatabaseofvariables.Satellitescancreateanddoread/writeoperationsonvariables.Foreachvariableitsname,type(floatingpoint,integer,string)anditscurrentvalueisstoredandmanaged.Variablescanbescalarvariablesorarrays,inwhichcasealsothenumberofdimensionsandsizeofthedimensionsisstored.Thesesizescanchangedynamically.

Actasaneventnotificationmanager.Satellitescansubscribetoasetofpredefinedevents.Forexample,ifasatellitesubscribestomutationeventsonaparticularvariable,theDataManagerwillsendanotificationtothatsatellitewheneverthevariableisupdated.

5

ResearcherSatelliteASatelliteBDataManagerSatelliteCSatelliteDSatellite...Figure3:DataManagerandSatellites

ThefunctionalityoftheDataManagerispurposelylimited:itcanbeusedbysatellitesforcommunication,butitdoesnotcontrolthesesatellitesthemselves.Thisisincontrasttodatafloworientedapplicationbuilders,inwhichtheunderlyingexecutionmodelsdictatewhenmoduleswillbefired[16,11,17].Anumberofsmallbutusefulgeneralpurposesatelliteshavebeendeveloped.WiththesesatellitesvariablesintheDataManagercanbeupdated.Someexamplesare:

dmdump,dmrestoredumpandrestorethevaluesofthevariablestoandfromfile;

dmsliceselectssubsetsofarrays;

dmlogmaintainsalogofthelast

valuesofavariable;

dmcalcevaluatesarithmeticexpressionsonvariables;

dmschemeisaSchemeinterpreterwhichisextendedwiththeAPItotheDataManager;

ReadDMandWriteDMaretwoIRISExplorermoduleswhichtranslateDataManagervariablesintotheExplorerdatatypescxParameterorcxLatticeandback.

CommunicationofasatellitewiththeDataManagerisdoneviaasmallApplicationProgrammersInterface(API).TheabstractionsusedbythislowlevelAPIaresimilartostandardUNIXI/Ofilehandling,withvariablesinsteadoffiles.Satellitesusehandlestoread,sampleandwritetovariables.Asamplecallreturnsimmediately,areadcallwaitsuntilthevalueofavariablechanges.EventsareusedtoindicatestatechangesintheDataManager.VariousroutinesareprovidedtoquerythestatusofvariablesandtheDataManager.

ThefunctionalityprovidedbythislowlevelAPIiscompact,terseandcomplete,butnotsimpletouse.Therefore,ontopofthisAPIaDataI/Olibrarywasdefined,whichistunedtotheneedsofresearchersthatwanttointegratetheirsimulationswithintheCSE.ThetwomaincallsaredioConnecttoregisteravariablewhosevaluemustbereadand/orwritten,anddioUpdate,whichcalltakescareofupdatingthevariablesinthesimulation.Entrypointsinexistingcodewherethesecallsmustbeaddedaregenerallyeasytolocate:connectionofvariablesjustbeforethemainloop,andupdateofvariablesattheendofthemainloop,justafterthenewresultsof

6

thetime-stephavebeencalculated.Theapplicationprogrammerdoesnothavetochangethecontrol-flowofhissimulation,andcanuseaproceduralprogramminglanguage.

3ParametrizedGraphicsObjects

3.1Overview

Inthissectionatoolforthegraphicalinteractionofaresearcherwithasimulationisdescribed.Asstatedbefore,animportantrequirementforsuchatoolisthatthevisualizationofthedata,thehandlingofuserinput,aswellasdirectmanipulationareprovided.Onewaytosolvethisistoconsidertheseaspectsasdisjoint,andtoprovideingenious,butunrelatedsolutions.Wehavechosenadifferentsolution:lookforthegreatestcommondivisoroftheseaspects,andprovideanhomogeneoussolution.Thegreatestcommondivisorofuserinputwidgetsandvisualizationtoolsissimplygraphics.Buttons,sliders,graphs,histogramsallboildowntocollectionsofgraphicsobjects.Therefore,weuseParametrizedGraphicsObjects(PGOs)asthemainconcept,andthegraphicseditorasametaphorforthedesignoftheinterface.

Thegraphicseditorhastwomodes:specificationandapplication,orshorter,editandrun.Inedit-mode,theresearchercancreateandeditgraphicsobjectsmuchlikeinMacDraw-likedrawingeditors.ThepropertiesofthoseobjectscanbeparametrizedtovaluesofvariablesintheDataManager.Hence,theresearcherdrawsaspecificationoftheinterface.Inrun-mode,atwowaycommunicationisestablishedbetweentheresearcherandthesimulationbybindingthesepropertiestovariables.Dataisretrievedandmappedontothepropertiesofthegraphicsobjects.Theresearchercanentertext,dragandpickobjects,whichistranslatedintochangesofthevaluesofvariables.Whatyoudrawiswhatyoucontrol.Theworkingmethodofaresearcheristhus:1.Decidewhichparametersareimportantforcontrolandvisualization;2.AdaptthesimulationtoconnectthoseparameterswiththeDataManager;3.Editaninterface;

4.Runtheinterface:viewandcontrolthesimulation;5.Analyzetheresultsandgobacktooneoftheprevioussteps.

Inaddition,standardsatellitescanbeusedornewsatellitescanbedeveloped.Thegraphicseditoritselfisjustasatellite,asshowninfigure4.WhentheresearcherinteractswithPGOs,datawillbewrittentotheDataManager.Similarly,writesbyothersatellitestovariableswilltriggerthegraphicseditorandresultinvisualizationsofthedata.

Figure5showsthattheresearcherisfreetodesignrepresentationsthatconveythesemanticsoftheunderlyingvariables:NinedifferentwaystovisualizetwoscalarvariablesandareshownBoththeedit-andtherun-modeversionsoftheinterfacearegiven.ThevariablescanbepresentedviastandardUI-widgets(aandb),orinabusinessgraphicsstyle(c).Theparameterscanalsorefertoaposition(d),arange(e),orhaveamechanical(f)orsensual(g)interpretation.Typicalcomputergraphicsinterpretationsaregivenin(h)and(i).Manymorerepresentationscanbeconceived.

Figure6providesanoverviewofthemainobjectswithinthePGOeditor.Foreachobjectoneormoreexamplesoftheirpresentationtotheresearcherareshown.VariousParametrized

7

ResearchertextdragpickanimationGraphicsBindingDatadataPGOeditoreventsSatellite...DataManagerFigure4:ThePGOeditorintheCSE

GraphicsObjects(PGOs)areshowninthetoprowofthefigure.Thegeometryofeachoftheseobjectsisdefinedbypoints,thenon-geometricpropertiesaredefinedbyvariousattributes.Inthebottomrowofthefiguretheobjectsforlocaldatamanagementareshown.TheobjectresponsibleforthebindingbetweengraphicsanddataistheDegreeOfFreedom,showninthemiddlerow.

Inthefollowingsubsectionswefirstdescribeeachobjectinmoredetail.Afterthat,wewilldescribehowtheyinteractwhentheeditorisswitchedtorun-mode.Finally,thehandlingofarraysisdiscussed.Applicationsaregiveninthenextsection.

3.2Graphicsobjects

ThePGOeditoroffersasetofstandardgraphicsobjects:fill-area,polyline,rectangle,circle,arc,andtext.Thegeometryoftheobjectsisdefinedbyoneormorepoints.Thesepointsareindependentofthegraphicsobjectsthemselves,sothatonepointcanbesharedbyvariousgraphicsobjects.

Theresearchercandefinerelationshipsbetweenpoints(figure7).Anypointcanhaveanotherpointasareference-pointorparent-point.Theserelationsareshownasgreylineswithyellowarrows.Cyclesarenotallowed,thus,thestructureisaforestoftrees,withpointsasnodesandleaves.Theserelationsareusedwhenpointsaremoved.Howthisisdonedependsonthetypeofthepoint.Twotypesofpointscanbeused:

Hingepoints(depictedascircles,andlabeledH).Whenahingepointismoved,thesametranslationisappliedtoitschildpoints.

Fixedpoints(depictedasdiamonds,andlabeledF).Ifafixedpointismoved,thenitschildrenarerotatedsuchthattheanglesbetweenthepointsandtheirdistancestotheparentpointremainfixed.

Nextthesetransformationsarerecursivelyappliedtothechildrenofthetransformedpoints.Infigure7weshowtheeffectofmovingapointinrun-modealongaDegreeofFreedom.

Graphicsobjectshavefourattributes:thehue,saturation,andvalueofitscolor,andthelinewidthusedfortheobjectoritsoutline.IntextobjectsreferencestoMappedVariablescanbemadebyusinga$,followedbyaMappedVariablename.Inrun-modethisreferenceisreplacedbythevalueofthecorrespondingvariable.

8

yxX = $xyxyxyxY = $yayxxxyxybcdyxxxxexfghiedit-mode

X = 0.659Y = 0.388abcdefghirun-mode

Figure5:Ninerepresentationsoftwoscalarvariables

9

GraphicsGraphicsobjectGraphicsobjectyxxGraphicsobjectGraphicsobjectGraphicsobjectzp=$xPointxAttributesWidthHueSat.ValuezxPress:setxpto1to0Release:setxpxtempyBindingDegreeofFreedomxminimumvaluecurrentvaluemaximumvaluemappedvariableDataMappedVariableNameMapVariableInputxpositionPXH12OutputMapNamepositionVariableNameTypeValuePXH12float46.2Minimum0.0Maximum100.0#decimals1Figure6:ObjectsinthePGOeditor

10

HHFFFFabFHHFFigure7:Pointsandpointstructures

3.3DegreesofFreedom

DegreesofFreedom(DOFs)areusedtoparametrizepointsandattributesasafunctionofvariables.DOFshaveastandardvisualrepresentationwhichisshowninfigure6.EachDOFhasaminimum,amaximum,acurrentvalue,andpossiblyaMappedVariablethatisboundtotheDOF.Inedit-modeallaspectsoftheDOFscanbechangedinteractivelyviadraggingandtext-editing,inrun-modeonlythecurrentvaluecanbechanged.SevenoptionsareavailableforassigningDOFstoapoint:noDOFs,CartesianDOFsineither,,orboth,andpolarDOFsinradius,angleorboth(seefigure8).PolarDOFscanonlybeusedforpointswithparent-points.

rxyyaxraFigure8:Degreesoffreedomforpoints

Withtheseoptionsincombinationwiththerelationsbetweenpointsawidevarietyofcoordinateframescanbedefined:relative,absolute,Cartesian,polar,hierarchical,althoughtheconceptofacoordinateframeitselfisnotprovided.Thisisatypicalexampleofthemainprinciplethathasguidedus:provideasetofsimpleprimitivesandusefuloperationstocombinethem,sothattheusercaneasilyconstructawiderangeofhigherorderconcepts.

DOFscanalsobeusedfortheattributeshue,saturation,value,andlinewidth,inthesamewayasDOFswereusedtoassociateamappedvariablewithgeometry.Theseattributesarepresentedtotheresearcherviaaseparateattributewindow(figure6).WiththoseDOFsatmostfourvariablevaluescanbevisualizedsimultaneously.

11

3.4MappedVariables

ThenamethataresearchercanspecifyforaDOFreferstoaMappedVariable(MVAR).ThedataofaMVARarereferencestoaMapandaVariable,andtwoon/offswitchesforinputandoutput.Withtheseswitchestheresearchercanselectifinputinformation(fromtheresearchertotheDataManager)andoutputinformation(fromtheDataManagertotheresearcher)canpassornot.

TheMapcontainsaspecificationhowvaluesmustbemappedinthecommunicationwiththeDataManager.Thecurrentimplementationissimple:onlylinearmappingsaresupported,hencethespecificationofaminimumandmaximumvaluesuffices.Furthermore,theMapcontainsaspecificationoftheformatfortextualoutput.AVariableisalocalcopyoftheinformationintheDataManager.Heresomebookkeepinginformation,suchasthetype,size,nameandDataManagerid,andthecurrentvalue(s)arestored.

Asimplerimplementationwouldbetousejustoneobjectthatincludesallinformation.However,theindirectionviatheMVARsisveryuseful.SeveralVariablescansharethesameMaps,thus,tochangethemappingofthosevariablesonlyasinglemaphastobechanged.Asanexample,supposethatasimulationcalculatesavarietyofwatertemperatures(1,2,...),allwithinthesamerange(say,0to100C).Herewecanusethesamemap()forallvariables.Also,onevariablecanhaveseveralassociatedmappings.Forinstance,awidemappingtoselectavarietyofvalues,andanarrowonetovisualizeifavalueisaboveorbelowathresholdvalue.Asanexample,tovisualizeifwaterisboilingornot,wecanuseanadditional

from99.5to100.5Celcius.map

3.5Output

Intheprevioussectionswehavedescribedtheobjectsinthesystem,nowwecandescribehow

theyinteractinrun-mode.Westartwithvisualization:theroutefromdataintheDataManagertoimagesandanimations.

Uponswitchingfromedittorun-modethePGOeditorfirstmakesaconnectiontotheDataManager.ForeachMVARthatisusedbyaDOF(orthatisreferencedinatextobject),andforwhichtheswitchoutputison,thePGOeditorexpressesinterestinmutationeventsfortheassociatedvariables.Thealgorithmforupdatingtheimageisasfollows:

1.Whenanotificationeventarrivesforavariable,itsvalue(say)issampledandstoredlocally.

2.Viathemap(say

)anormalizedvalue

iscalculated:

0

1

If

isoutsidetherangeofthemap,

isclamped.

3.ForallDOFsboundtoMVARsthecurrentvalue

isupdatedvia:

1

4.Thecorrespondingobjectsareupdated.Colorsarerecalculated:mappedfromHSVtoRGB-space.ThecoordinatesofpointsandalltheirchildrenareupdatedaccordingtothetypeofDOF.

12

5.Finally,thegraphicsobjectsareredrawn,usingthechangedpointsandthenewvaluesforattributes,andtherebydisplayingthechangesinthedata.ReferencestoMVARsintextobjectsarereplacedbytheformattedvalueofthevariable.RedrawingtheimageforeachchangeofthedataoftheDataManagerisexpensive,andcanleadtoerroneousanimations.Asanexample,consideranobjectwithcoordinatesand.Typicallyasimulationwillchangebothandinasingletime-step.Iftheimageisredrawnforeachchangeinthevalueofoneofthevariables,theobservedtrajectoryoftheobjectwillhaveastaircaseform.Therefore,theresearchercanspecifyatriggervariable.Ifatriggervariable(say)hasbeenspecified,thenthePGOeditoronlyawaitschangesin.Whenchanges,allrelevantvariablesintheDataManageraresampled,andtheimageisredrawn.

3.6Input

Theprocessingofuserinputinrun-modeisverysimilartothehandlingofoutput,exceptthatthedirectionofthedataflowisreversed.Pointscanbedragged,butnow,incontrasttoedit-mode,onlyalongthedirection(s)specifiedbytheassociatedDOFs.ChangesinthecorrespondingMVARsaredealtwithinthereversewayasdescribedintheprevioussubsection.Childpointsmovealongjustastheywouldinedit-mode:theDOFsmovealongwiththepoint.

Theresearchercanalsochangeattribute-valuesofgraphicsobjectsintheattributewindow,providedthatDOFshavebeendefinedforthem.ThisisdonebydraggingthecurrentvalueoftheDOFintheattributewindowtoanewvalue.

Pickingisausefulinteractionprimitivefortheselectionofobjectsandforinvokingactions.Wehavedefinedpickinginlinewiththedata-drivenconceptofthesystem:apickresultsinachangeofdata.Onestandardactionispredefined:Setthevalueofavariabletoavalue.Foreachgraphicsobjecttheresearchercanspecifywhichvariablehastobeassignedwhatvalue,bothforpress-events(amouse-buttonispressedwiththepointerinsidethegraphicsobjects)andforrelease-events(amouse-buttonisreleased).

Textobjectscanbechangedinrun-modebytheresearcherifthetextobjectcontainsoneormorereferencestoMVARs,andifthe’input’-switchesoftheMVARsareon.Theresearchercanclickatsuchanobject,thestringisreplacedbyaninputbox,andtheresearchercanenternewtext.Nextthistextisscanned,thevalueofthevariableassociatedwiththeMVARisupdated,andthisnewvalueissenttotheDataManager.Also,theimageisredrawntoshowupdatesinothergraphicsobjectsthatdependonthesamevariable.

3.7Arrays

Oftenthemajorpartofdatatobevisualizedwillbearrays.ThemanualspecificationofalargenumberofsimilarPGOs,withDOFsparametrizedasindexedelementsofthearray,isatediousprocess.Therefore,weautomatedit.Eachgraphicsobjectinedit-modeisconsideredasatemplate,fromwhichinstancesaregeneratedafterbindingwiththeDataManager.Fromallgraphicsobjectsmultipleinstancescanbegenerated.Inaddition,polylinesandfill-areaspointscanbeexpandedin-linewithinasingleobject(seefigure9).

13

bbbbbbbbbbbbaMultiple instancescaIn-line expansioncFigure9:Twodifferentexpansionsofarraysforpolylineobject

4Applications

4.1Balls

Infigure10anedit-modeversionoffigure2isshown.Weusedarray-expansionfortheballs.Onlyasinglecirclewithasinglevelocity-lineisdefined.Eachofthevariables,,,andisanarray.Themappedvariableisboundtothevariable,withtheinput-switchtooff.Thus,whenaballisdragged,theradiusisnotinfluenced.Thevariableisscalar.Ifwereanarray(eachballhasadifferentsize),stillthesameinterfacecouldbeused.Thecoloroftheenclosingboxisparametrizedtothedampingfactorofthevelocity.

Thestandardsatellitedmlogincombinationwitharray-expansionwasusedtomakethegraphofthetotalkineticenergyinthelower-leftcorner.Apolylineforasinglepointwithasuitableparametrizationwasdefined.Ifloggingisactive,thepointisexpandedinaseriesofpoints,andhenceaslidinggraphisdisplayed.Thisexampleshowsthatstandardscientificgraphicsprimitivescaneasilybedefined.

4.2Cars

Figure11showsthattheCSEcanbeusedformulti-dimensionalvisualization,andasafront-endtodatabases.WeusedourCSEtovisualizeatableof400differenttypesofcars,whereforeachcaranumberofattributessuchasdisplacement,horsepower,andaccelerationaregiven1.Wedevelopedasmallsatellitethatreadsinthedata,writesthesedatatotheDataManager,andthatcanbeusedtomakeselectionsofthedata.Thedataarevisualizedviathreescatter-plots.Eachiteminthescatter-plotrepresentsacartype.Foreachitemweusedcolortovisualizethecountryoforigin,andtwolinestoindicatemilespergallonandweight.Eachaxisofthescatter-plotshasanassociatedrange-slidertomakeselectionsofthecars.Further,theusercanclickateachitem,andthepropertiesofthiscartypeareshowninthelowerrightcorner.

pyhRadiusrw$rrrContact forcevyccFcdvxrcpx$ckContact dampingkFcdvelocityc$kkDampingFdkvvelocityTimeSynchronoust = $tdt = $dtEkin_tEkin = $EkinAsynchronousField forceFyFyFyFx$FyFx$kvkvAttractionFaaaaaaEkin_x$FxFx$aFigure10:Simulationofbouncingballs,edit-mode

15

455DISP.DISP.ACC.8.0ACC.24.86846HP.230230MPGUSAEUROPEWEIGHTHP.NAME : \"chevrolet malibu\"DISP : 350.0ACC : 13.0HP : 145JAPANWEIGHT : 3988CYLINDERS : 8YEAR : 73MPG : 1846Figure11:Visualfront-endtoamulti-dimensionaldatabase.

16

4.3Atmosphericsimulation

WehaveappliedourtechniquestoamodelforsmogpredictionoverEurope.Thefullblownmodelforecaststhelevelsofairpollution,whichischaracterizedbyapproximately104reactionsbetweenca.70species.Forexample,theconcentrationsofozone(3),sulphurdioxide(2)andsulphateaerosol(4)arecalculated.Thevericalstratificationismodeledbyfourlayers;thesurfacelayer,themixinglayer,thereservoirlayerandtheupperlayer.Thephysicalandchemicalmodelisdescribedbyasetofpartialdifferentialequationsthatdescribeadvection,diffusion,emission,wetanddrydeposition,fumigationandchemicalreactions.

Animportantnumericalutilitytosolvetheseequationsislocalgridrefinement.Thistechniqueisusedtoimprovethequalityofthemodelcalculationsinareaswithlargespatialgradients(forexampleinregionswithstrongemissions)andinareasofinterest,specifiedbytheresearcher.Thetradeofftobemadeinlocalgridrefinementiscalculationaccuracyversuscomputationspeed.

Wehaveusedourenvironmenttosteervariousaspectsofthelocalgridrefinementtechnique.Wenameonlyafewoftheseaspects:

theresearchermayselectthespeciesandthelayertowhichgridrefinementwillbeapplied.

defineathetolerancevaluewhichdetermineswhererefinementisnecessary.

defineaboundingboxasaregionofinteresttodrivethegridrefinement.

interactivecontroloversimulationtime.

Figure12showsasnapshotoftheongoingsimulation.Ontheleft,inthePGOeditor,onlytherefinedgridwithinaregionofinterestisshown.Visualizedisonespecies(inthiscase,ozone)atonelayerandthewindfield.Ontheright,intheIRISExplorerRendermodule,allfourlayers

tosenddatatoIRISExplorer.Theofthesamespeciesisshown.Weusedthesatellite

satelliteisshownonthelowerright.

Theamountofdataproducedpertimestepofthesimulationissubstantial.Foreachgridpointtheconcentrationofeachspeciesiscalculated.Dependingonthegridrefinementlevel,theamountofdatacanvarybetween1to4megabytespertimestep.Thereareapproximately500timestepsinourtestsimulations.

Thegeometryofthegrid-cellsandallconcentrationsarestoredasvariablesintheDataMan-ager.Thedmslicesatelliteisusedtointeractivelyselectaparticularconcentration.Concentrationsarevisualizedviaspecificationofasinglerectanglethatisparametrizedwiththesevariables.Thewindfieldisvisualizedviaspecificationofasinglepolyline.Boththerectangleandpolylinespecificationsareexpandedthroughthearrayexpansionmechanism.

5Discussion

Intheprevioussectionwehavepresentedapplications.Inthissectionwestepbacktoahigherlevelofabstraction.WediscusstheunderlyingconceptsofthedevelopedCSE,compareourworkwithrelatedwork,considerfutureextensionsandthescopeofthemethodsandtechniquesdescribed.

17

Figure12:VisualizationofasmogpredictionmodelwiththePGOeditorandIRISExplorer

18

5.1Concepts

ThepresentedCSEhastwomajorcomponents:theDataManagerandthePGO-editor.Severalconceptsaresharedbybothcomponents:

Theuseoflow-levelprimitives:asimpledatamodelandgraphicsobjects.Theinterfacestotheseprimitivesarefamiliartotheend-user:asimpleI/Olibraryfordatamanipulationandagraphicseditorforgraphics.

Nohigherlevelsemanticsaredefinedforthegraphicsandthedata.Asaresult,theenvironmentisgeneralandflexible.Theresearchercaneasilyaddmeaningtothegraphicsanddata.WiththePGOeditortheend-usercaneditandcombinePGOstorealizemeaningfulwidgets.Changestoexistingsimulationsoftwareareminimal.Thelocationswherehookfunctionsmustbeaddedareeasytolocate,datastructuresandprogramstructuresdonothavetobechanged.Summarizing:theCSEitselfisnotatoolfor,say,physicallybasedmodelingandanimation,butitcouldbeagreatenvironmentforthedevelopmentofsuchasystem.

BoththeDataManagerandthePGOeditorrelyonlatebindingofnames.Asaresult,itispossibletodefinenewvisualizationsofthedataoutputbythesimulation,whilethesimulationcontinuestorun.

AlloperationsinboththeDataManagerandthePGOeditorarebasedentirelyondata.Dragging,pickingandtextinputaretranslatedintochangesofdata.Themaintypeofeventwithinthesystemisthedatamutation.Processcontrol(firingalgorithmsorexecutionmodels)canbeimplementedontopofthoseeventsbythesatellites.

5.2Comparison

Insection1.3weconsideredthreeapproachestorealizeanenvironmentforcomputationalsteering.Ourapproachfallsintheclassofextensionofgraphicstool-kits.MostrelatedtoourworkarethereforetheInventor-systemandthe3-DinteractivesystembeingdevelopedatBrownUniversity.However,therearealsoimportantdifferences.Anobviousdifferenceisthatthosetwosystemsare3-D.Wewilldiscusstheextensionofoursystemfrom2-Dto3-Dinthenextsection.

Ourstartingpointwastheresearcherasanend-user:howcanweofferanenvironmentthatsuitshisneedsandiseasytouse?Thisisreflectedinthecontentsofthepaper.Weextensivelydiscussedhowtheenvironmentlooksfromtheend-userpointofview,andpaidlessattentiontothecomputersciencemethodsandtechniquesused.Thiswasadeliberatechoice:forusthiswasthemostimportantissue.TheapproachtakenbyInventorandatBrownUniversityisprimarilyofinterestforapplicationprogrammersandnotforend-users.

Anotherinterestingdifferenceisthehandlingofuser-inputandhence,directmanipulation.InbothInventorandatBrownUniversityseparateobjectsareusedforthis:adistinctionismadebetweengeometricobjectsandmanipulators.Relationsbetweenthosetwocategoriesmustbespecifiedviatext.ForthePGOsthisdistinctionwasnotmade.Intheirterms:eachgraphicsobjectcanserveasamanipulator.Feedbackviathemanipulatedobjectisalwaysprovided.Ifsuchamanipulationmusthaveaneffectonotherobjects,thentheuseofthesamenamesinDOFssufficesforlinearmappings.Ifmorecomplexbehaviorisrequired,thiscanbeaddedasaseparatesatellite.

19

5.3Futurework

ConcerningtheDataManager,wewillconsidertheuseofsharedmemoryandpoint-to-pointcommunicationschemesformoreefficientinterprocesscommunication.Inthecurrentimple-mentationawritewillalwaystransportdatatotheDataManagerwhich,inturn,willtransportthedatatoallsubscribingsatellites.Thisindirectmethodofcommunicationcanbeprohibitivelyexpensiveforlargedatasets.Thisprobleminthecurrentimplementationwillbeovercomebyallowingpoint-to-pointchannelsbetweensatellites.Inthisway,theDataManagerwillactonlyasarouterforlargedata-sets.

Inthecurrentenvironmentallgraphicsaretwo-dimensional.Thisallowedarapiddevelopmentoftheenvironment,andanearlyvalidationoftheunderlyingconcepts.Two-dimensionalPGOsdonotrendertheCSEuseless.Manyreal-timesimulationsaretwo-orevenone-dimensional,sothata2-DPGOeditorismorethanjustified.Further,theresultsofthree-dimensionalsimulationscanbemappedontwo-dimensionaldata(slices),ordataoflowerdimensionality.Finally,wehaveshownthatwecanincludeexistingtoolsfor3-Dvisualizationinourenvironment.Nevertheless,a3-DversionofthePGOswouldbewelcome.Thiswouldmeanthatwehavetoimplementsolutionsforviewing,directmanipulationandcursorpositioningin3-D.Butthatisprimarilyaproblemrelatedto3-D,andnottoourenvironment.Weexpectthatthebasicconcepts,andespeciallytheone-dimensionalDOFs,canbetransposedto3-Dwithoutmajorproblems.

WehavetailoredourenvironmenttoComputationalSteering(seefigure1).Itisaninterestingexercisetoreplaceresearcherbyuserandsimulationbyapplication.Weexpectthattheenvi-ronmentanditsunderlyingconceptswillbeusefulformanyotherapplicationareas.Wehopetoinvestigatethisinthefuture.Someexamplesofpossibleapplicationsofourvisualspecificationtechniques:customwidgetsinUserInterfaceManagementSystems;interactiveanimationandsimulationforComputerAidedEducationandhypermedia;andtemplate-drivengraphicseditors.

6Conclusions

anarchitecturewithacentralDataManager;

Wehaveshownthatthecombineduseof:

parametrizedgraphicsobjects;

resultsinanenvironmentthatprovides

easyintegrationofdifferentprocesses;

easydefinitionofvisualizations;

easydefinitionofcustomizeduser-interfacewidgets;

directmanipulationbydefault.

Theprimarytargetapplicationfortheenvironmentiscomputationalsteering,butthesameen-vironment,andatleastthesameconcepts,canprobablybeappliedinmanyotherapplicationareas.Inshort,wehavedevelopedasystemthatmakesiteasyforanend-usertospecifyanduseinteractivecomputergraphics.

20

References

[1]B.McCormick,T.Defanti,andM.Brown.VisualizationinScientificComputing.Computer

Graphics,22(6(SIGGRAPH’88)):103–111,1987.[2]R.E.Marshall,J.L.Kempf,D.ScottDyer,andC-CYen.VisualizationMethodsandSimula-tionSteeringa3DTurbulenceModelofLakeErie.1990Symp.onInteractive3DGraphics,ComputerGraphics,24(2):–97,1990.[3]P.S.StraussandR.Carey.AnObject-Oriented3DGraphicsToolkit.ComputerGraphics,

26(6(SIGGRAPH’92)):341–351,1992.[4]A.vanDam.VRasaForcingFunction:SoftwareImplicationsofaNewParadigm.IEEE

VirtualRealitySymposiumProceedings,pages6–9,1993.[5]R.Zeleznik,D.BrookshireConner,M.Wloka,D.Aliaga,N.Huang,P.Hubbard,B.Knep,

H.Kaufman,J.Hughes,andA.vanDam.AnObject-OrientedFrameworkfortheIntegrationofInteractiveAnimationTechniques.ComputerGraphics,25(4(SIGGRAPH’91)):105–111,1991.[6]S.Snibbe,K.Herndon,D.Robbins,D.BrookshireConner,andA.vanDam.UsingDeforma-tionstoExplore3DWidgetDesign.ComputerGraphics,26(6(SIGGRAPH’92)):351–353,1992.[7]B.Myers.UserInterfaceTools:IntroductionandSurvey.IEEESoftware,6(1):15–23,19.[8]D.OlsenJr.WorkshoponSoftwareToolsforUserInterfaceManagement.Computer

Graphics,21(2):71–147,1987.[9]G.Pfaff,editor.UserInterfaceManagementSystems.SpringerVerlag,1985.

[10]B.Myers.CreatingUserInterfacesUsingProgrammingbyExample,VisualProgramming,

andConstraints.ACMTransactionsonProgrammingLanguagesandSystems,12(2):143–177,1990.[11]C.Upson,T.Faulhaber,D.Kamins,D.Laidlaw,D.Schelgel,J.Vroom,R.Gurwitz,and

A.vanDam.TheApplicationVisualizationSystem:AComputationalEnvironmentforScientificVisualization.IEEEComputerGraphicsandApplications,9(7):30–42,19.[12]C.Upson.VolumetricVisualizationTechniques.InD.F.RogersandR.AEarnshaw,editors,

StateoftheArtinComputerGraphics,pages313–350.SprigerVerlag,1991.[13]W.FelgerandF.Schroder.TheVisualizationInputPipeline–EnablingSemanticInteraction

inScientificVisualization.ComputerGraphicsForum,II(3):139–152,1992.[14]L.Bergman,J.Richardson,D.Richardson,andF.BrooksJr.VIEW–AnExploratory

MolecularVisualizationSystemwithUser-DefinableInteractionSequences.ComputerGraphics,27(6(SIGGRAPH’93)):117–126,1993.[15]J.J.vanWijkandR.vanLiere.AnEnvironmentforComputationalSteering.Technical

ReportCS-R9450,CentreforMathematicsandInformatics,Amsterdam,TheNetherlands,1993.

21

[16]IRISExplorerDevelopmentTeam.IRISExplorerModuleWriter’sGuide.Technical

Report007-1369-020,SiliconGraphicsComputerSystems,Mtn.View,CA.,1993.

[17]C.Williams,J.Rasure,andC.Hansen.TheStateoftheArtofVisualLanguagesfor

Visualization.InProceedingsVisualization’92,pages202–209,1992.

22