Matthias Traub, Rene Kaiser and Wolfgang Weiss
Institute of Information and Communication Technologies
email@example.com, firstname.lastname@example.org, email@example.com
As information technology has advanced, the usage of online social networks rapidly increased at the same time, and social interaction shifted onto these platforms. Online social networks have gained significant popularity and are now among the most popular sites on the Web. Likewise, sophisticated video communication systems, which in the past mainly concentrated on the business videoconferencing market, are shifting into the private domain. In the realm of these developments, we are aiming for integration of online social networks with intelligent video communication systems. Relevant information extracted from massive amounts of online social network data could inform intelligent behaviour within video communication systems, and therefore needs to be extracted. This article presents a selection of taxonomies of relationships as basis for multidimensional social networks that we consider relevant for such purposes. As users might not necessarily be very active, this selection also considers how well it could work with sparse network layers. Based on a literature study, we conclude that the combination of layers with sparse information onto a multidimensional network usually increases the usefulness of the data for its analysis.
Social networks data is mainly composed out of social relationships. These relations are created by either by users, or by the online social network platform itself automatically. People who interact with each other, e.g. by sharing common activities or being member of a common group, form a social network. Users can establish groups of common interests, interact and collaborate with one another and create conscious (explicit) or unconscious (implicit) relationships. The social network makes the ways in which users are related explicit, and thereby allows to analyse how they are connected, based on one or several relation types . Implicit relations can be extracted from online social network sites with state-of-the art algorithms.
The relationships in an online social network can be categorized in many different ways, based upon different taxonomies that reflect certain aspects of the relation. Each of these categories represents additional information that can be exploited to analyse the relations’ characteristics and their dynamics. These taxonomies must be carefully examined regarding the purpose of the analysis. Once the data is extracted, the categorization creates a consistent and overarching view on each relationship. Exploiting this additional information about each relationship, in contrast to utilizing only the basic information about the relation (e.g. actor, object, relation type, weight), can be a big advantage in terms of network analysis.
A social network analysis algorithm could focus for example on communication between users, and as a first step, it would create a social network based on direct relations between users. On the other hand the analysis could also be based on common activities like sharing or liking an image, or commenting on the same blog post, which would allow extracting indirect relationships. Merging multiple such layers, possibly both more topically-related and socially-related links, into a multidimensional social network can be more useful than examining online social networks containing only sparse information .
The analysis’ purpose that serves as context for the considerations of this article is to extract useful information from an online social network that can feed into the social network’s video communication system. The more the latter knows about the participants of the current session, the better it can satisfy the individual needs and preferences of its users.
Which pairs of users of the videoconferencing session have the strongest social ties? Which of them have a common interest about the topic that is currently being discussed (speech to text based analysis)? Which users within the group have a special static role, like the teacher in a session together with pupils? Knowing about the participants’ communication behaviour regarding text-based interaction in the social network (cp. ), could we predict their behaviour in video communication, e.g. who is shy and who will dominate the conversation by speaking all the time?
One aspect in video communication systems that could utilize such information is communication orchestration [14, 10]. This concept refers to an automatic process that decides which of the available video streams in a group video communication session are displayed for each particular users, and how. Especially in larger groups or setups with multiple cameras, orchestration can help users to see (and hear) what’s most relevant to them at each point in time, supporting the users’ individual communication needs. Putting the right video streams on screen in proper size, based automatic understanding on verbal and non-verbal interaction, enables to follow the dynamic conversation and to read facial expressions and subtle communication cues. In that sense, communication orchestration can exploit answers to the example questions in the previous paragraph.
A concrete example: if the videoconferencing system has access to all profiles of the session’s participants, it could derive static roles like “teacher” and “pupils” in this setup. The visual presentation could hence adapt, supporting the teacher with a good overview of the group in terms of camera selection and presentation, and supporting the pupils by giving the teacher a special, more important role, keeping him or her visible on screen most of the time, even when another pupil would be speaking. For further details on communication orchestration, please refer to  and .This article is structured as follows. In Section II we discuss related work from the field of social network analysis, in particular such with connection to communication behaviour in online social networks and such that is generating social networks out of different feature layers. Then, we highlight basic information about the taxonomies of relationships and the specification of multidimensional social networks. We conclude a literature research by proposing and discussing a selection of relevant taxonomies with the use-case of informing communication orchestration in mind.
2 RELATED WORK
Social network analysis has long been an interesting area of study in the fields of human computer interaction, sociology and computer supported collaborative work. In general, online social networks display a rich internal structure where users can create relations of various types and mostly decide which and how many relations to create.
Those networks have existed since the beginning of the Internet. The graph derived from email communication can be used to form a social network. These communication networks have been studied by Diesner  in 2005 where email traffic of 21,000 employees was analysed. Dynamics of the structure and properties of the organizational communication network were explored, as well as the dynamics of the characteristics and patterns of communicative behaviour of the employees from different organizational levels.
Social networks illustrate the relation and connection between users and, as mentioned before, these connections or ties between users can be based on to either one or several relations. If those relations are weighted, the network can be treated as a fuzzy network .Multimedia sharing sites like Flickr already have been subject of many social network studies. New relations emerge from the users’ common tags. Direct links between contact lists were also studied. Based on a Flickr dataset, Kazienko  defined nine types of relations between users. These relations were extracted from the contact lists, tags, groups of items and their authors, favourite pictures and comments to pictures. Some of them, like favourites and comments, were then split into three sub layers, e.g. author-commentator, commentator-author and commentator-commentator.
3 TAXONOMY OF RELATIONSHIPS
A relationship in an online social network can be clustered in many different ways and based on various characteristics. In this section we give a short overview of selected taxonomies we propose to utilize for supporting intelligent video conferencing tools that are integrated with online social networks. These taxonomies are useful for the observation of the relations between two users. In certain cases, the relation between two users is defined by a common relation to a third object, which we refer to as meeting object in the following. The following is based on previous work by the authors of , , , , and .
An obvious classification is based on the active subject that created a certain relationship. It can be divided into three main types:
Further, relationship dynamics like durability and intensity of a relationship can be predicted to a certain degree based on the relationship origin. Determining the creator of a relationship leads to a classification of users who are very active in creating new relations .
Another aspect of social relationships is the awareness of the users involved within a relation:
awareness typically correlates with the level of activity within the network.
If the relations of a network are constructed mainly out of relationships where
both users are aware of it, chances are higher that these networks survive. In terms of relationship
dynamics, the awareness factor is also important . Do relationships behave
differently when they start off created by the system or when both users are
aware of the relation right away?
As another relevant taxonomy, the direction and mutuality of the connection between users can be divided into three types:
in the directions of relationships can be used to analyse the relationship
dynamics, e.g. to what extent and when do asymmetrical relations change to
symmetrical, and vice versa .
This taxonomy is based on the type of data used for relationship creation. Splitting relationships into categories based on the type of data used to create those relations, adds additional granularity level to the analysis. Based on , we divided the relationships resting upon the following three compound data types:
categorization enables the investigation of the correlation between these
features and type of data used to create a relationship. Considering the
observation of just a single activity, e.g. the commenting of a photo, the
information has limited usefulness for the analysis of the whole network. For example the relation was created because
two users commented the same photo but after that they just exchange direct
messages. Only larger sets of data bring insights into the relationship
dynamics (cp. ).
The directness of the relationship provides more useful information about the users and their interaction than other categories. There are three kinds of directness of a relationship :
Fig. 1. Direct relation with unidirectional connections. There could also be direct bidirectional relations.
Fig. 2. Quasi-direct relation with eaual roles.
Fig. 3. Quasi-direct relation with different roles.
4 MULTIDIMENSIONAL SOCIAL NETWORKS
A social network created out of the users’ communication behaviour consists of a single layer. However, a social network extracted from a typical online social network is based on a range of individual features available from users. The set of connections and relationships is derived directly from data about user activities, e.g. comments on blog posts, liking and sharing pictures, follower and followee links. Each feature may bind users in a different way and forms another kind of relation. Using the relationship taxonomies described above for every applicable feature creates a certain social network layer.
A connection between user A and B exists if at least one relation of any type is present. All of them combined create a strongly related group of users. Considering an online social network with only sparse information on a particular group of users, the observation of only one relationship feature would not give useful data to analyse .
For better understanding, Figure 4 shows a simple illustration of a multidimensional network and its combination of different layers. The bottom layer (resulting network) is a combination of three network layers based on different taxonomies and aspects. The figure contains five different users A to E displayed in bordered rectangles and meeting objects 1 to 3 displayed in double bordered green boxes. In the meeting object creator layer, the direct connection between meeting objects and their creator is illustrated. Since there are three meeting objects, there must be at least one, up to a maximum three, creators connected to them. The communication connection layer shows the user-to-user direct communication connection. In our example, user A follows user B, and E follows D. The meeting object commentator layer shows the connection between the meeting objects and the creation of comments. The combination of those three layers is shown in the bottom layer.The combined multidimensional layer gives a complete overview of the whole network. It contains all the previous connections and enables more complex analysis methods. For example, in none of the other layers a connection from user A to D was clearly evident. Combining layers with sparse information can create a network with dense connections and serves as a basis for typical network analysis methods .
Fig. 4. Combination of feature extraction layers to a combined multidimensional social network layer. User entities are display in colored rectangles labelled A to E. Meeting objects are depicted as green double borderd rectangles. The arrows illustrate the relation between the entities and objects.
In this article, we discuss possibilities for information extraction out of an online social network environment. The use-case we had in mind was to extract information that could support a real-time video communication system [10, 14] which exploits users’ social media profiles and streams as background knowledge for orchestrating video communication. Especially information about the communication behaviour inside the network with the other participants of the video conference session could be interesting for orchestration purposes, as well as shared common activities and similarities of user profiles, such as common interests.
Information about the users’ connections inside the online social network can be used to inform communication orchestration and network optimization components of a video communication system. This could bias camera selection in order to preferably show other users that are strongly connected to the current user, which may especially support users in heated discussions of larger groups where it is difficult to keep the overview. General classification of user profiles can be used to specify communication preferences. Orchestration may even influence the conversation as a moderator through its actions, for example supporting shy participants to participate by giving them visual presence easier than others once they decide to actively communicate.
This work is part of an ongoing research activity. So far we have not managed to implement a dedicated social network analysis component that is integrated with our video communication system. Therefore, this article does not include any evaluation of the construction of a multidimensional social network and the usefulness of the presented taxonomies for our application.
The limitation of each of the selected taxonomies in isolation is that they can be used to analyse only one feature of the relationships between users. To create a complete description of a certain relationship, it is essential to define a category within each of the selected taxonomies. Utilizing this extra information implies benefits for the analysis. Only the compound set of relationships should be used for creating a layer for the multidimensional social network observation.Our literature research has further confirmed that the combination of multiple relation types based on different social network features also gives more insight into the global social network, especially if the existing data is sparse.